Note: This is test shard 4 of 8.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from LocationAwareRebalancingBasicTest
[ RUN ] LocationAwareRebalancingBasicTest.Basic
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250112 21:06:01.721928 26953 test_util.cc:274] Using random seed: -1309055940
I20250112 21:06:01.726652 26953 ts_itest-base.cc:112] Starting cluster with:
I20250112 21:06:01.726693 26953 ts_itest-base.cc:113] --------------
I20250112 21:06:01.726715 26953 ts_itest-base.cc:114] 6 tablet servers
I20250112 21:06:01.726734 26953 ts_itest-base.cc:115] 3 replicas per TS
I20250112 21:06:01.726753 26953 ts_itest-base.cc:116] --------------
2025-01-12T21:06:01Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-12T21:06:01Z Disabled control of system clock
I20250112 21:06:01.734225 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:44733
--webserver_interface=127.26.82.126
--webserver_port=0
--builtin_ntp_servers=127.26.82.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.82.126:44733
--location_mapping_cmd=/tmp/dist-test-taskam1Qel/build/release/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/location-assignment.state --map /A:2 --map /B:2 --map /C:2
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20250112 21:06:01.806396 26965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:01.806555 26965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:01.806573 26965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:01.807834 26965 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250112 21:06:01.807878 26965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250112 21:06:01.807893 26965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:01.807905 26965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250112 21:06:01.807922 26965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250112 21:06:01.809260 26965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskam1Qel/build/release/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/location-assignment.state --map /A:2 --map /B:2 --map /C:2
--ipki_ca_key_size=768
--master_addresses=127.26.82.126:44733
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:44733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.82.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.26965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:01.809415 26965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:01.809576 26965 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:01.813269 26970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:01.813421 26971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:01.814333 26973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:01.815537 26965 server_base.cc:1034] running on GCE node
I20250112 21:06:01.815721 26965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:01.815934 26965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:01.817055 26965 hybrid_clock.cc:648] HybridClock initialized: now 1736715961817039 us; error 28 us; skew 500 ppm
I20250112 21:06:01.823338 26965 webserver.cc:458] Webserver started at http://127.26.82.126:41765/ using document root <none> and password file <none>
I20250112 21:06:01.823498 26965 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:01.823535 26965 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:01.823603 26965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:01.824364 26965 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c"
format_stamp: "Formatted at 2025-01-12 21:06:01 on dist-test-slave-929r"
I20250112 21:06:01.824590 26965 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c"
format_stamp: "Formatted at 2025-01-12 21:06:01 on dist-test-slave-929r"
I20250112 21:06:01.825781 26965 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:01.826406 26979 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:01.826586 26965 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:01.826658 26965 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c"
format_stamp: "Formatted at 2025-01-12 21:06:01 on dist-test-slave-929r"
I20250112 21:06:01.826716 26965 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:01.848443 26965 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:01.848762 26965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:01.848876 26965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:01.852180 26965 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.126:44733
I20250112 21:06:01.852223 27031 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.126:44733 every 8 connection(s)
I20250112 21:06:01.852465 26965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250112 21:06:01.852986 27032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:01.855160 27032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c: Bootstrap starting.
I20250112 21:06:01.855811 27032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:01.856045 27032 log.cc:826] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:01.856572 27032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c: No bootstrap required, opened a new log
I20250112 21:06:01.857712 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 26965
I20250112 21:06:01.857802 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250112 21:06:01.857986 27032 raft_consensus.cc:357] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 44733 } }
I20250112 21:06:01.858090 27032 raft_consensus.cc:383] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:01.858109 27032 raft_consensus.cc:738] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ffc7d3d1d84bcea2dd92a719b8fe5c, State: Initialized, Role: FOLLOWER
I20250112 21:06:01.858220 27032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 44733 } }
I20250112 21:06:01.858271 27032 raft_consensus.cc:397] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 21:06:01.858294 27032 raft_consensus.cc:491] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 21:06:01.858333 27032 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:01.858831 27032 raft_consensus.cc:513] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 44733 } }
I20250112 21:06:01.858947 27032 leader_election.cc:304] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52ffc7d3d1d84bcea2dd92a719b8fe5c; no voters:
I20250112 21:06:01.859090 27032 leader_election.cc:290] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 21:06:01.859148 27037 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:01.859334 27037 raft_consensus.cc:695] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [term 1 LEADER]: Becoming Leader. State: Replica: 52ffc7d3d1d84bcea2dd92a719b8fe5c, State: Running, Role: LEADER
I20250112 21:06:01.859385 27032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [sys.catalog]: configured and running, proceeding with master startup.
I20250112 21:06:01.860040 27037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 44733 } }
I20250112 21:06:01.860605 27038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52ffc7d3d1d84bcea2dd92a719b8fe5c. Latest consensus state: current_term: 1 leader_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 44733 } } }
I20250112 21:06:01.860592 27039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ffc7d3d1d84bcea2dd92a719b8fe5c" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 44733 } } }
I20250112 21:06:01.860720 27038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:01.860724 27039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:01.861012 27050 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 21:06:01.861605 27050 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 21:06:01.862933 27050 catalog_manager.cc:1348] Generated new cluster ID: 9b574a085a194f3585a101509b73d69d
I20250112 21:06:01.862987 27050 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 21:06:01.882153 27050 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 21:06:01.882731 27050 catalog_manager.cc:1505] Loading token signing keys...
I20250112 21:06:01.891654 27050 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 52ffc7d3d1d84bcea2dd92a719b8fe5c: Generated new TSK 0
I20250112 21:06:01.891785 27050 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 21:06:01.894145 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.65:0
--local_ip_for_outbound_sockets=127.26.82.65
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--builtin_ntp_servers=127.26.82.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20250112 21:06:01.966660 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:01.966917 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:01.966985 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:01.969193 27056 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250112 21:06:01.969250 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:01.969287 27056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.65
I20250112 21:06:01.971546 27056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--never_fsync=true
--heap_profile_path=/tmp/kudu.27056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.65
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:01.971864 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:01.972131 27056 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:01.977458 27062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:01.978338 27064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:01.978535 27061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:01.979418 27056 server_base.cc:1034] running on GCE node
I20250112 21:06:01.979578 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:01.979765 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:01.980885 27056 hybrid_clock.cc:648] HybridClock initialized: now 1736715961980870 us; error 35 us; skew 500 ppm
I20250112 21:06:01.982215 27056 webserver.cc:458] Webserver started at http://127.26.82.65:33667/ using document root <none> and password file <none>
I20250112 21:06:01.982420 27056 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:01.982463 27056 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:01.982581 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:01.983624 27056 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "90f636b130e64c04a7f6cdad4839ce33"
format_stamp: "Formatted at 2025-01-12 21:06:01 on dist-test-slave-929r"
I20250112 21:06:01.983971 27056 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "90f636b130e64c04a7f6cdad4839ce33"
format_stamp: "Formatted at 2025-01-12 21:06:01 on dist-test-slave-929r"
I20250112 21:06:01.986624 27056 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20250112 21:06:01.987426 27070 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:01.987648 27056 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:01.987723 27056 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "90f636b130e64c04a7f6cdad4839ce33"
format_stamp: "Formatted at 2025-01-12 21:06:01 on dist-test-slave-929r"
I20250112 21:06:01.987778 27056 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:02.006912 27056 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:02.007097 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:02.007197 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:02.007354 27056 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:02.007611 27056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:02.007642 27056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.007674 27056 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:02.007706 27056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.012903 27056 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.65:40025
I20250112 21:06:02.012964 27183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.65:40025 every 8 connection(s)
I20250112 21:06:02.013216 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250112 21:06:02.017468 27184 heartbeater.cc:346] Connected to a master server at 127.26.82.126:44733
I20250112 21:06:02.017556 27184 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:02.017719 27184 heartbeater.cc:510] Master 127.26.82.126:44733 requested a full tablet report, sending...
I20250112 21:06:02.018375 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 27056
I20250112 21:06:02.018441 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250112 21:06:02.020956 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.66:0
--local_ip_for_outbound_sockets=127.26.82.66
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--builtin_ntp_servers=127.26.82.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250112 21:06:02.049499 26996 ts_manager.cc:194] Registered new tserver with Master: 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:02.050295 26996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.65:56345
W20250112 21:06:02.094714 27188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:02.094862 27188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:02.094885 27188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:02.096167 27188 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250112 21:06:02.096210 27188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:02.096232 27188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.66
I20250112 21:06:02.097628 27188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--never_fsync=true
--heap_profile_path=/tmp/kudu.27188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.66
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:02.097813 27188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:02.098012 27188 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:02.102018 27194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.102792 27193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.103427 27196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:02.104166 27188 server_base.cc:1034] running on GCE node
I20250112 21:06:02.104331 27188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:02.104516 27188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:02.105640 27188 hybrid_clock.cc:648] HybridClock initialized: now 1736715962105612 us; error 41 us; skew 500 ppm
I20250112 21:06:02.106813 27188 webserver.cc:458] Webserver started at http://127.26.82.66:45111/ using document root <none> and password file <none>
I20250112 21:06:02.107067 27188 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:02.107131 27188 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:02.107234 27188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:02.108186 27188 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c60c223c1269463181610419f08e11fc"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.108446 27188 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c60c223c1269463181610419f08e11fc"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.109549 27188 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20250112 21:06:02.110239 27202 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.110423 27188 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:02.110496 27188 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c60c223c1269463181610419f08e11fc"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.110559 27188 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:02.131780 27188 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:02.131994 27188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:02.132092 27188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:02.132264 27188 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:02.132521 27188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:02.132555 27188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.132586 27188 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:02.132614 27188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.137745 27188 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.66:43929
I20250112 21:06:02.137780 27315 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.66:43929 every 8 connection(s)
I20250112 21:06:02.138036 27188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250112 21:06:02.142179 27316 heartbeater.cc:346] Connected to a master server at 127.26.82.126:44733
I20250112 21:06:02.142253 27316 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:02.142392 27316 heartbeater.cc:510] Master 127.26.82.126:44733 requested a full tablet report, sending...
I20250112 21:06:02.145267 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 27188
I20250112 21:06:02.145351 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250112 21:06:02.146422 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.67:0
--local_ip_for_outbound_sockets=127.26.82.67
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--builtin_ntp_servers=127.26.82.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250112 21:06:02.174093 26996 ts_manager.cc:194] Registered new tserver with Master: c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:02.174571 26996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.66:43149
W20250112 21:06:02.221362 27320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:02.221520 27320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:02.221550 27320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:02.222738 27320 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250112 21:06:02.222779 27320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:02.222815 27320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.67
I20250112 21:06:02.224148 27320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--never_fsync=true
--heap_profile_path=/tmp/kudu.27320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.67
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:02.224334 27320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:02.224536 27320 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:02.228986 27326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.229712 27325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.230036 27328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:02.230816 27320 server_base.cc:1034] running on GCE node
I20250112 21:06:02.230950 27320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:02.231134 27320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:02.232257 27320 hybrid_clock.cc:648] HybridClock initialized: now 1736715962232244 us; error 33 us; skew 500 ppm
I20250112 21:06:02.233290 27320 webserver.cc:458] Webserver started at http://127.26.82.67:40041/ using document root <none> and password file <none>
I20250112 21:06:02.233456 27320 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:02.233497 27320 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:02.233585 27320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:02.234364 27320 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.234661 27320 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.236233 27320 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20250112 21:06:02.236873 27334 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.237056 27320 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:02.237120 27320 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.237180 27320 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:02.247684 27320 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:02.247856 27320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:02.247946 27320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:02.248100 27320 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:02.248344 27320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:02.248375 27320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.248529 27320 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:02.248571 27320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.254047 27320 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.67:44103
I20250112 21:06:02.254112 27447 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.67:44103 every 8 connection(s)
I20250112 21:06:02.254357 27320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250112 21:06:02.259033 27448 heartbeater.cc:346] Connected to a master server at 127.26.82.126:44733
I20250112 21:06:02.259106 27448 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:02.259280 27448 heartbeater.cc:510] Master 127.26.82.126:44733 requested a full tablet report, sending...
I20250112 21:06:02.261159 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 27320
I20250112 21:06:02.261224 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250112 21:06:02.262557 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.68:0
--local_ip_for_outbound_sockets=127.26.82.68
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--builtin_ntp_servers=127.26.82.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250112 21:06:02.292937 26996 ts_manager.cc:194] Registered new tserver with Master: dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:02.293397 26996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.67:56979
W20250112 21:06:02.336540 27452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:02.336722 27452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:02.336746 27452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:02.338275 27452 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250112 21:06:02.338328 27452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:02.338369 27452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.68
I20250112 21:06:02.339720 27452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--never_fsync=true
--heap_profile_path=/tmp/kudu.27452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.68
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:02.339876 27452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:02.340032 27452 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:02.343852 27457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.344625 27460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.345325 27458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:02.346109 27452 server_base.cc:1034] running on GCE node
I20250112 21:06:02.346248 27452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:02.346521 27452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:02.347690 27452 hybrid_clock.cc:648] HybridClock initialized: now 1736715962347663 us; error 36 us; skew 500 ppm
I20250112 21:06:02.348753 27452 webserver.cc:458] Webserver started at http://127.26.82.68:38903/ using document root <none> and password file <none>
I20250112 21:06:02.348937 27452 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:02.348984 27452 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:02.349068 27452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:02.349931 27452 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.350206 27452 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.351317 27452 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:02.351891 27466 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.352023 27452 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:02.352089 27452 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.352146 27452 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:02.375828 27452 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:02.376020 27452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:02.376120 27452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:02.376281 27452 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:02.376531 27452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:02.376562 27452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.376595 27452 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:02.376626 27452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.381912 27452 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.68:43591
I20250112 21:06:02.381945 27579 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.68:43591 every 8 connection(s)
I20250112 21:06:02.382220 27452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250112 21:06:02.386234 27580 heartbeater.cc:346] Connected to a master server at 127.26.82.126:44733
I20250112 21:06:02.386317 27580 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:02.386334 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 27452
I20250112 21:06:02.386417 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250112 21:06:02.386511 27580 heartbeater.cc:510] Master 127.26.82.126:44733 requested a full tablet report, sending...
I20250112 21:06:02.387334 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.69:0
--local_ip_for_outbound_sockets=127.26.82.69
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--builtin_ntp_servers=127.26.82.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250112 21:06:02.417380 26995 ts_manager.cc:194] Registered new tserver with Master: efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:02.417827 26995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.68:45173
W20250112 21:06:02.459300 27583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:02.459437 27583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:02.459465 27583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:02.460638 27583 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250112 21:06:02.460688 27583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:02.460722 27583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.69
I20250112 21:06:02.462040 27583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--never_fsync=true
--heap_profile_path=/tmp/kudu.27583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.69
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:02.462230 27583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:02.462435 27583 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:02.466279 27589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.466980 27590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.467584 27592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:02.468739 27583 server_base.cc:1034] running on GCE node
I20250112 21:06:02.468886 27583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:02.469066 27583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:02.470194 27583 hybrid_clock.cc:648] HybridClock initialized: now 1736715962470172 us; error 38 us; skew 500 ppm
I20250112 21:06:02.471215 27583 webserver.cc:458] Webserver started at http://127.26.82.69:45139/ using document root <none> and password file <none>
I20250112 21:06:02.471390 27583 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:02.471433 27583 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:02.471524 27583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:02.472308 27583 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "9b470022ea814aa5bba1c6716c187a59"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.472580 27583 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "9b470022ea814aa5bba1c6716c187a59"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.473719 27583 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:02.474289 27598 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.474428 27583 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:02.474494 27583 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "9b470022ea814aa5bba1c6716c187a59"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.474550 27583 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:02.483433 27583 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:02.483603 27583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:02.483698 27583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:02.483839 27583 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:02.484077 27583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:02.484113 27583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.484144 27583 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:02.484174 27583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.489598 27583 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.69:38955
I20250112 21:06:02.489631 27711 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.69:38955 every 8 connection(s)
I20250112 21:06:02.489878 27583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250112 21:06:02.490937 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 27583
I20250112 21:06:02.491012 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20250112 21:06:02.491950 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.70:0
--local_ip_for_outbound_sockets=127.26.82.70
--webserver_interface=127.26.82.70
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--builtin_ntp_servers=127.26.82.84:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250112 21:06:02.494396 27712 heartbeater.cc:346] Connected to a master server at 127.26.82.126:44733
I20250112 21:06:02.494472 27712 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:02.494632 27712 heartbeater.cc:510] Master 127.26.82.126:44733 requested a full tablet report, sending...
I20250112 21:06:02.531013 26995 ts_manager.cc:194] Registered new tserver with Master: 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:02.531510 26995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.69:57573
W20250112 21:06:02.564555 27715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:02.564713 27715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:02.564738 27715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:02.566022 27715 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250112 21:06:02.566066 27715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:02.566102 27715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.70
I20250112 21:06:02.567471 27715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.82.70
--webserver_port=0
--tserver_master_addrs=127.26.82.126:44733
--never_fsync=true
--heap_profile_path=/tmp/kudu.27715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.70
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:02.567667 27715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:02.567860 27715 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:02.571604 27721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.572305 27722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:02.572889 27724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:02.573913 27715 server_base.cc:1034] running on GCE node
I20250112 21:06:02.574107 27715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:02.574313 27715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:02.575471 27715 hybrid_clock.cc:648] HybridClock initialized: now 1736715962575461 us; error 23 us; skew 500 ppm
I20250112 21:06:02.576568 27715 webserver.cc:458] Webserver started at http://127.26.82.70:36397/ using document root <none> and password file <none>
I20250112 21:06:02.576783 27715 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:02.576843 27715 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:02.576947 27715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:02.577764 27715 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "4aa042eb06ac407fa827c735ea972020"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.578050 27715 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "4aa042eb06ac407fa827c735ea972020"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.579043 27715 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:02.579564 27730 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.579697 27715 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:02.579762 27715 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "4aa042eb06ac407fa827c735ea972020"
format_stamp: "Formatted at 2025-01-12 21:06:02 on dist-test-slave-929r"
I20250112 21:06:02.579819 27715 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:02.601452 27715 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:02.601647 27715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:02.601752 27715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:02.601923 27715 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:02.602224 27715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:02.602257 27715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.602289 27715 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:02.602315 27715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:02.607527 27715 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.70:39441
I20250112 21:06:02.607591 27843 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.70:39441 every 8 connection(s)
I20250112 21:06:02.607831 27715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20250112 21:06:02.612357 27844 heartbeater.cc:346] Connected to a master server at 127.26.82.126:44733
I20250112 21:06:02.612437 27844 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:02.612612 27844 heartbeater.cc:510] Master 127.26.82.126:44733 requested a full tablet report, sending...
I20250112 21:06:02.615901 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 27715
I20250112 21:06:02.615975 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20250112 21:06:02.644032 26995 ts_manager.cc:194] Registered new tserver with Master: 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:02.644491 26995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.70:36831
I20250112 21:06:02.649778 26953 external_mini_cluster.cc:934] 6 TS(s) registered with all masters
I20250112 21:06:02.655711 26953 external_mini_cluster.cc:934] 6 TS(s) registered with all masters
I20250112 21:06:03.051191 27184 heartbeater.cc:502] Master 127.26.82.126:44733 was elected leader, sending a full tablet report...
I20250112 21:06:03.175594 27316 heartbeater.cc:502] Master 127.26.82.126:44733 was elected leader, sending a full tablet report...
I20250112 21:06:03.294164 27448 heartbeater.cc:502] Master 127.26.82.126:44733 was elected leader, sending a full tablet report...
I20250112 21:06:03.418802 27580 heartbeater.cc:502] Master 127.26.82.126:44733 was elected leader, sending a full tablet report...
I20250112 21:06:03.532361 27712 heartbeater.cc:502] Master 127.26.82.126:44733 was elected leader, sending a full tablet report...
I20250112 21:06:03.645352 27844 heartbeater.cc:502] Master 127.26.82.126:44733 was elected leader, sending a full tablet report...
I20250112 21:06:06.406692 26995 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:06.416122 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 66f307b250124c46980183887a4b460d (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.416122 27646 tablet_service.cc:1467] Processing CreateTablet for tablet 66f307b250124c46980183887a4b460d (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.416442 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f307b250124c46980183887a4b460d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.416442 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f307b250124c46980183887a4b460d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.416769 27381 tablet_service.cc:1467] Processing CreateTablet for tablet 525c56c9e1d442a89ef2f6428fc0949d (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.418483 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 66f307b250124c46980183887a4b460d (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.418762 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f307b250124c46980183887a4b460d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.418720 27118 tablet_service.cc:1467] Processing CreateTablet for tablet c2c3d077653440c88798984259616078 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.418968 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2c3d077653440c88798984259616078. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.419306 27645 tablet_service.cc:1467] Processing CreateTablet for tablet 525c56c9e1d442a89ef2f6428fc0949d (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.419366 27869 tablet_bootstrap.cc:492] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:06.419394 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 525c56c9e1d442a89ef2f6428fc0949d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.420145 27869 tablet_bootstrap.cc:654] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.420415 27869 log.cc:826] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:06.421439 27871 tablet_bootstrap.cc:492] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:06.421470 27778 tablet_service.cc:1467] Processing CreateTablet for tablet c2c3d077653440c88798984259616078 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.421721 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2c3d077653440c88798984259616078. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.421836 27117 tablet_service.cc:1467] Processing CreateTablet for tablet 525c56c9e1d442a89ef2f6428fc0949d (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.421921 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 525c56c9e1d442a89ef2f6428fc0949d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.422048 27871 tablet_bootstrap.cc:654] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.422326 27871 log.cc:826] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:06.422701 27869 tablet_bootstrap.cc:492] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:06.422757 27869 ts_tablet_manager.cc:1397] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:06.423219 27381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 525c56c9e1d442a89ef2f6428fc0949d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.424824 27875 tablet_bootstrap.cc:492] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:06.424872 27872 tablet_bootstrap.cc:492] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:06.424857 27869 raft_consensus.cc:357] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.425091 27869 raft_consensus.cc:383] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.425182 27869 raft_consensus.cc:738] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.425127 27514 tablet_service.cc:1467] Processing CreateTablet for tablet c2c3d077653440c88798984259616078 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5c2249ca2e943829ff96b6aae07c90c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.425371 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2c3d077653440c88798984259616078. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.425455 27872 tablet_bootstrap.cc:654] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.425467 27875 tablet_bootstrap.cc:654] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.425374 27869 consensus_queue.cc:260] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.425657 27869 ts_tablet_manager.cc:1428] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:06.425738 27875 log.cc:826] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:06.425879 27872 log.cc:826] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:06.426415 27875 tablet_bootstrap.cc:492] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:06.426445 27872 tablet_bootstrap.cc:492] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:06.426453 27869 tablet_bootstrap.cc:492] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:06.426494 27872 ts_tablet_manager.cc:1397] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:06.426528 27875 ts_tablet_manager.cc:1397] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:06.426927 27869 tablet_bootstrap.cc:654] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.427556 27874 tablet_bootstrap.cc:492] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:06.428352 27874 tablet_bootstrap.cc:654] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.428483 27871 tablet_bootstrap.cc:492] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:06.428578 27871 ts_tablet_manager.cc:1397] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250112 21:06:06.428647 27874 log.cc:826] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:06.428822 27875 raft_consensus.cc:357] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.428822 27872 raft_consensus.cc:357] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.428951 27872 raft_consensus.cc:383] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.428982 27872 raft_consensus.cc:738] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.429098 27872 consensus_queue.cc:260] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.429143 27875 raft_consensus.cc:383] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.429178 27875 raft_consensus.cc:738] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.429343 27872 ts_tablet_manager.cc:1428] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:06.429309 27875 consensus_queue.cc:260] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.429533 27875 ts_tablet_manager.cc:1428] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:06.429630 27869 tablet_bootstrap.cc:492] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:06.429674 27869 ts_tablet_manager.cc:1397] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:06.429800 27869 raft_consensus.cc:357] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.429852 27869 raft_consensus.cc:383] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.429869 27869 raft_consensus.cc:738] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.429909 27869 consensus_queue.cc:260] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.429976 27869 ts_tablet_manager.cc:1428] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.430600 27872 tablet_bootstrap.cc:492] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:06.430845 27874 tablet_bootstrap.cc:492] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:06.430909 27874 ts_tablet_manager.cc:1397] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:06.431028 27871 raft_consensus.cc:357] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.431124 27872 tablet_bootstrap.cc:654] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.431165 27871 raft_consensus.cc:383] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.431196 27871 raft_consensus.cc:738] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.431190 27880 tablet_bootstrap.cc:492] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:06.431298 27871 consensus_queue.cc:260] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.431533 27871 ts_tablet_manager.cc:1428] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:06.431743 27880 tablet_bootstrap.cc:654] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.431826 27872 tablet_bootstrap.cc:492] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:06.431872 27872 ts_tablet_manager.cc:1397] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:06.432060 27872 raft_consensus.cc:357] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.432135 27872 raft_consensus.cc:383] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.432125 27880 log.cc:826] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:06.432157 27872 raft_consensus.cc:738] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.432200 27872 consensus_queue.cc:260] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.432288 27872 ts_tablet_manager.cc:1428] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.432969 27880 tablet_bootstrap.cc:492] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:06.432901 27874 raft_consensus.cc:357] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.433008 27874 raft_consensus.cc:383] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.433027 27880 ts_tablet_manager.cc:1397] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:06.433040 27874 raft_consensus.cc:738] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.433115 27874 consensus_queue.cc:260] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.433321 27874 ts_tablet_manager.cc:1428] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20250112 21:06:06.433387 27874 tablet_bootstrap.cc:492] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:06.433780 27874 tablet_bootstrap.cc:654] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.434366 27874 tablet_bootstrap.cc:492] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:06.434422 27874 ts_tablet_manager.cc:1397] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:06.434572 27874 raft_consensus.cc:357] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.434649 27874 raft_consensus.cc:383] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.434682 27874 raft_consensus.cc:738] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.434738 27874 consensus_queue.cc:260] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.434844 27874 ts_tablet_manager.cc:1428] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.435418 27880 raft_consensus.cc:357] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.435534 27880 raft_consensus.cc:383] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.435556 27880 raft_consensus.cc:738] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.435613 27880 consensus_queue.cc:260] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.435766 27880 ts_tablet_manager.cc:1428] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:06.471066 27887 raft_consensus.cc:491] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:06.471148 27887 raft_consensus.cc:513] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.471395 27887 leader_election.cc:290] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:06.473863 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "525c56c9e1d442a89ef2f6428fc0949d" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:06.473913 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "525c56c9e1d442a89ef2f6428fc0949d" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:06.473968 27666 raft_consensus.cc:2463] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:06.474033 27138 raft_consensus.cc:2463] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:06.474184 27336 leader_election.cc:304] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:06.474325 27887 raft_consensus.cc:2798] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:06.474378 27887 raft_consensus.cc:491] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:06.474401 27887 raft_consensus.cc:3054] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.474915 27887 raft_consensus.cc:513] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.475015 27887 leader_election.cc:290] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Requested vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:06.475185 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "525c56c9e1d442a89ef2f6428fc0949d" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:06.475235 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "525c56c9e1d442a89ef2f6428fc0949d" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:06.475262 27138 raft_consensus.cc:3054] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.475296 27666 raft_consensus.cc:3054] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.475808 27138 raft_consensus.cc:2463] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:06.475869 27666 raft_consensus.cc:2463] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:06.475963 27337 leader_election.cc:304] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:06.476090 27887 raft_consensus.cc:2798] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:06.476220 27887 raft_consensus.cc:695] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 LEADER]: Becoming Leader. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Running, Role: LEADER
I20250112 21:06:06.476300 27887 consensus_queue.cc:237] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:06.476871 26995 catalog_manager.cc:5526] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } }
W20250112 21:06:06.492352 27713 tablet.cc:2367] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250112 21:06:06.507001 27449 tablet.cc:2367] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250112 21:06:06.515846 27185 tablet.cc:2367] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:06.542711 27876 raft_consensus.cc:491] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:06.542788 27876 raft_consensus.cc:513] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.543067 27876 leader_election.cc:290] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:06.545699 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f307b250124c46980183887a4b460d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:06.545794 27402 raft_consensus.cc:2463] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:06.545953 27599 leader_election.cc:304] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:06.546082 27876 raft_consensus.cc:2798] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:06.546080 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f307b250124c46980183887a4b460d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:06.546136 27876 raft_consensus.cc:491] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:06.546164 27876 raft_consensus.cc:3054] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.546216 27270 raft_consensus.cc:2463] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:06.546797 27876 raft_consensus.cc:513] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.546908 27876 leader_election.cc:290] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Requested vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:06.547051 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f307b250124c46980183887a4b460d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:06.547104 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f307b250124c46980183887a4b460d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:06.547122 27270 raft_consensus.cc:3054] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.547168 27402 raft_consensus.cc:3054] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.547693 27270 raft_consensus.cc:2463] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:06.547780 27402 raft_consensus.cc:2463] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:06.547869 27599 leader_election.cc:304] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:06.547953 27876 raft_consensus.cc:2798] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:06.548069 27876 raft_consensus.cc:695] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [term 1 LEADER]: Becoming Leader. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Running, Role: LEADER
I20250112 21:06:06.548159 27876 consensus_queue.cc:237] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.548792 26995 catalog_manager.cc:5526] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69). New cstate: current_term: 1 leader_uuid: "9b470022ea814aa5bba1c6716c187a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:06.606701 27883 raft_consensus.cc:491] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:06.606804 27883 raft_consensus.cc:513] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.607115 27883 leader_election.cc:290] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
W20250112 21:06:06.610352 27845 tablet.cc:2367] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:06.610342 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c3d077653440c88798984259616078" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:06.610481 27534 raft_consensus.cc:2463] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:06.610585 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c3d077653440c88798984259616078" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:06.610668 27138 raft_consensus.cc:2463] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:06.610698 27731 leader_election.cc:304] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:06.610802 27883 raft_consensus.cc:2798] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:06.610864 27883 raft_consensus.cc:491] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:06.610886 27883 raft_consensus.cc:3054] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.611459 27883 raft_consensus.cc:513] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.611569 27883 leader_election.cc:290] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:06.611711 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c3d077653440c88798984259616078" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:06.611739 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c3d077653440c88798984259616078" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:06.611770 27534 raft_consensus.cc:3054] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.611802 27138 raft_consensus.cc:3054] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.612287 27138 raft_consensus.cc:2463] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:06.612365 27534 raft_consensus.cc:2463] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:06.612476 27733 leader_election.cc:304] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, 90f636b130e64c04a7f6cdad4839ce33; no voters:
I20250112 21:06:06.612561 27883 raft_consensus.cc:2798] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:06.612704 27883 raft_consensus.cc:695] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Running, Role: LEADER
I20250112 21:06:06.612782 27883 consensus_queue.cc:237] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:06.613438 26995 catalog_manager.cc:5526] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4aa042eb06ac407fa827c735ea972020 (127.26.82.70). New cstate: current_term: 1 leader_uuid: "4aa042eb06ac407fa827c735ea972020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } }
W20250112 21:06:06.634701 27581 tablet.cc:2367] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250112 21:06:06.640976 27317 tablet.cc:2367] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250112 21:06:06.716780 27902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:06.716939 27902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:06.718153 27902 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20250112 21:06:06.750998 27138 raft_consensus.cc:1270] T c2c3d077653440c88798984259616078 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:06.751027 27534 raft_consensus.cc:1270] T c2c3d077653440c88798984259616078 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:06.751340 27900 consensus_queue.cc:1035] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:06.751588 27883 consensus_queue.cc:1035] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:06.751616 27138 raft_consensus.cc:1270] T 525c56c9e1d442a89ef2f6428fc0949d P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:06.751677 27666 raft_consensus.cc:1270] T 525c56c9e1d442a89ef2f6428fc0949d P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:06.751935 27887 consensus_queue.cc:1035] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:06.752143 27887 consensus_queue.cc:1035] T 525c56c9e1d442a89ef2f6428fc0949d P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:06.755870 27935 mvcc.cc:204] Tried to move back new op lower bound from 7113588599810441216 to 7113588598687293440. Current Snapshot: MvccSnapshot[applied={T|T < 7113588599810441216}]
I20250112 21:06:06.756330 27936 mvcc.cc:204] Tried to move back new op lower bound from 7113588599812268032 to 7113588599246278656. Current Snapshot: MvccSnapshot[applied={T|T < 7113588599812268032}]
I20250112 21:06:06.756548 27937 mvcc.cc:204] Tried to move back new op lower bound from 7113588599812268032 to 7113588599246278656. Current Snapshot: MvccSnapshot[applied={T|T < 7113588599808020480}]
I20250112 21:06:06.757627 27943 mvcc.cc:204] Tried to move back new op lower bound from 7113588599810441216 to 7113588598687293440. Current Snapshot: MvccSnapshot[applied={T|T < 7113588599810441216 or (T in {7113588599810441216})}]
I20250112 21:06:06.758285 27402 raft_consensus.cc:1270] T 66f307b250124c46980183887a4b460d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:06.758291 27270 raft_consensus.cc:1270] T 66f307b250124c46980183887a4b460d P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:06.758533 27896 consensus_queue.cc:1035] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:06.758728 27876 consensus_queue.cc:1035] T 66f307b250124c46980183887a4b460d P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:06.759825 27934 mvcc.cc:204] Tried to move back new op lower bound from 7113588599810441216 to 7113588598687293440. Current Snapshot: MvccSnapshot[applied={T|T < 7113588599810441216}]
INSERT report
rows total: 2000
time total: 30.5099 ms
time per row: 0.0152549 ms
I20250112 21:06:06.771613 27954 mvcc.cc:204] Tried to move back new op lower bound from 7113588599842369536 to 7113588598981668864. Current Snapshot: MvccSnapshot[applied={T|T < 7113588599842369536}]
I20250112 21:06:06.771767 26993 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:06.773401 27514 tablet_service.cc:1467] Processing CreateTablet for tablet 2736e58da0e64362960fc8cc8612ecd2 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.773447 27513 tablet_service.cc:1467] Processing CreateTablet for tablet 377842729046491aa373c2c9244804c9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.773480 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 377842729046491aa373c2c9244804c9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.773532 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2736e58da0e64362960fc8cc8612ecd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.773555 27778 tablet_service.cc:1467] Processing CreateTablet for tablet 2736e58da0e64362960fc8cc8612ecd2 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.773603 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377842729046491aa373c2c9244804c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.773640 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377842729046491aa373c2c9244804c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.773658 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2736e58da0e64362960fc8cc8612ecd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.774778 27880 tablet_bootstrap.cc:492] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:06.774863 27646 tablet_service.cc:1467] Processing CreateTablet for tablet e60af3902e844f088d271b897635e987 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.774894 27875 tablet_bootstrap.cc:492] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:06.774947 27777 tablet_service.cc:1467] Processing CreateTablet for tablet 377842729046491aa373c2c9244804c9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.774967 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e60af3902e844f088d271b897635e987. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.775028 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377842729046491aa373c2c9244804c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.775224 27880 tablet_bootstrap.cc:654] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.775331 27875 tablet_bootstrap.cc:654] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.775653 27871 tablet_bootstrap.cc:492] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:06.776050 27871 tablet_bootstrap.cc:654] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.776083 27869 tablet_bootstrap.cc:492] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:06.773614 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 2736e58da0e64362960fc8cc8612ecd2 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.776449 27869 tablet_bootstrap.cc:654] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.776465 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2736e58da0e64362960fc8cc8612ecd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.776481 27118 tablet_service.cc:1467] Processing CreateTablet for tablet e60af3902e844f088d271b897635e987 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.776595 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e60af3902e844f088d271b897635e987. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.776729 27381 tablet_service.cc:1467] Processing CreateTablet for tablet e60af3902e844f088d271b897635e987 (DEFAULT_TABLE table=rebalance_test_table_1 [id=3b055565600d4b86920a79f603b733ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:06.776844 27381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e60af3902e844f088d271b897635e987. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:06.777076 27880 tablet_bootstrap.cc:492] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:06.777117 27880 ts_tablet_manager.cc:1397] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:06.777276 27880 raft_consensus.cc:357] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.777343 27880 raft_consensus.cc:383] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.777366 27880 raft_consensus.cc:738] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.777415 27880 consensus_queue.cc:260] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.777494 27880 ts_tablet_manager.cc:1428] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:06.777559 27880 tablet_bootstrap.cc:492] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:06.777949 27880 tablet_bootstrap.cc:654] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.778067 27872 tablet_bootstrap.cc:492] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:06.778465 27871 tablet_bootstrap.cc:492] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:06.778465 27875 tablet_bootstrap.cc:492] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:06.778506 27871 ts_tablet_manager.cc:1397] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:06.778506 27875 ts_tablet_manager.cc:1397] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250112 21:06:06.778627 27869 tablet_bootstrap.cc:492] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:06.778669 27869 ts_tablet_manager.cc:1397] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:06.778647 27875 raft_consensus.cc:357] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.778656 27871 raft_consensus.cc:357] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.778681 27872 tablet_bootstrap.cc:654] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.778704 27871 raft_consensus.cc:383] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.778704 27875 raft_consensus.cc:383] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.778725 27875 raft_consensus.cc:738] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.778724 27871 raft_consensus.cc:738] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.778764 27871 consensus_queue.cc:260] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.778764 27875 consensus_queue.cc:260] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.778837 27871 ts_tablet_manager.cc:1428] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.778807 27869 raft_consensus.cc:357] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.778849 27875 ts_tablet_manager.cc:1428] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.778872 27869 raft_consensus.cc:383] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.778892 27871 tablet_bootstrap.cc:492] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:06.778903 27875 tablet_bootstrap.cc:492] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:06.778892 27869 raft_consensus.cc:738] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.778954 27869 consensus_queue.cc:260] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.779064 27869 ts_tablet_manager.cc:1428] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.779081 27874 tablet_bootstrap.cc:492] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:06.779294 27871 tablet_bootstrap.cc:654] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.779301 27875 tablet_bootstrap.cc:654] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.779376 27874 tablet_bootstrap.cc:654] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:06.779958 27871 tablet_bootstrap.cc:492] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:06.780022 27871 ts_tablet_manager.cc:1397] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:06.780148 27871 raft_consensus.cc:357] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.780211 27871 raft_consensus.cc:383] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.780236 27871 raft_consensus.cc:738] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.780289 27871 consensus_queue.cc:260] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.780390 27871 ts_tablet_manager.cc:1428] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:06.780426 27872 tablet_bootstrap.cc:492] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:06.780467 27872 ts_tablet_manager.cc:1397] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:06.780639 27872 raft_consensus.cc:357] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.780738 27872 raft_consensus.cc:383] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.780767 27872 raft_consensus.cc:738] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.780798 27875 tablet_bootstrap.cc:492] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:06.780815 27872 consensus_queue.cc:260] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.780897 27875 ts_tablet_manager.cc:1397] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:06.780925 27872 ts_tablet_manager.cc:1428] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:06.781060 27875 raft_consensus.cc:357] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.781126 27875 raft_consensus.cc:383] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.781150 27875 raft_consensus.cc:738] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.781194 27880 tablet_bootstrap.cc:492] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:06.781189 27875 consensus_queue.cc:260] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.781244 27880 ts_tablet_manager.cc:1397] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250112 21:06:06.781272 27875 ts_tablet_manager.cc:1428] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.781369 27874 tablet_bootstrap.cc:492] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:06.781379 27880 raft_consensus.cc:357] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.781410 27874 ts_tablet_manager.cc:1397] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:06.781435 27880 raft_consensus.cc:383] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.781457 27880 raft_consensus.cc:738] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.781502 27880 consensus_queue.cc:260] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.781599 27880 ts_tablet_manager.cc:1428] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.781575 27874 raft_consensus.cc:357] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.781644 27874 raft_consensus.cc:383] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:06.781666 27874 raft_consensus.cc:738] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:06.781702 27874 consensus_queue.cc:260] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:06.781792 27874 ts_tablet_manager.cc:1428] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:06.791471 27889 raft_consensus.cc:491] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:06.791548 27889 raft_consensus.cc:513] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.791803 27889 leader_election.cc:290] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:06.794698 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377842729046491aa373c2c9244804c9" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:06.794778 27270 raft_consensus.cc:2463] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:06.794948 27467 leader_election.cc:304] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:06.794982 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377842729046491aa373c2c9244804c9" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:06.795065 27798 raft_consensus.cc:2463] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:06.795073 27889 raft_consensus.cc:2798] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:06.795125 27889 raft_consensus.cc:491] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:06.795143 27889 raft_consensus.cc:3054] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.795643 27889 raft_consensus.cc:513] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.795740 27889 leader_election.cc:290] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Requested vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:06.795887 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377842729046491aa373c2c9244804c9" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:06.795948 27270 raft_consensus.cc:3054] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.796440 27270 raft_consensus.cc:2463] T 377842729046491aa373c2c9244804c9 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:06.796533 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377842729046491aa373c2c9244804c9" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:06.796589 27467 leader_election.cc:304] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:06.796605 27798 raft_consensus.cc:3054] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:06.796674 27889 raft_consensus.cc:2798] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:06.796790 27889 raft_consensus.cc:695] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 LEADER]: Becoming Leader. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Running, Role: LEADER
I20250112 21:06:06.796890 27889 consensus_queue.cc:237] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:06.797278 27798 raft_consensus.cc:2463] T 377842729046491aa373c2c9244804c9 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:06.797446 26993 catalog_manager.cc:5526] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 reported cstate change: term changed from 0 to 1, leader changed from <none> to efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68). New cstate: current_term: 1 leader_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:07.082036 27889 raft_consensus.cc:491] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:07.082140 27889 raft_consensus.cc:513] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:07.082321 27889 leader_election.cc:290] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:07.082516 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2736e58da0e64362960fc8cc8612ecd2" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:07.082533 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2736e58da0e64362960fc8cc8612ecd2" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:07.082593 27798 raft_consensus.cc:2463] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:07.082630 27270 raft_consensus.cc:2463] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:07.082752 27468 leader_election.cc:304] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.082868 27889 raft_consensus.cc:2798] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:07.082908 27889 raft_consensus.cc:491] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:07.082926 27889 raft_consensus.cc:3054] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.083570 27889 raft_consensus.cc:513] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:07.083683 27889 leader_election.cc:290] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Requested vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:07.083842 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2736e58da0e64362960fc8cc8612ecd2" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:07.083854 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2736e58da0e64362960fc8cc8612ecd2" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:07.083904 27270 raft_consensus.cc:3054] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.083917 27798 raft_consensus.cc:3054] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.084435 27798 raft_consensus.cc:2463] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:07.084444 27270 raft_consensus.cc:2463] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:07.084599 27468 leader_election.cc:304] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.084702 27889 raft_consensus.cc:2798] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:07.084766 27889 raft_consensus.cc:695] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 LEADER]: Becoming Leader. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Running, Role: LEADER
I20250112 21:06:07.084827 27889 consensus_queue.cc:237] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:07.085345 26993 catalog_manager.cc:5526] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 reported cstate change: term changed from 0 to 1, leader changed from <none> to efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68). New cstate: current_term: 1 leader_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:07.157723 27896 raft_consensus.cc:491] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:07.157789 27896 raft_consensus.cc:513] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.157948 27896 leader_election.cc:290] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:07.158115 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60af3902e844f088d271b897635e987" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:07.158218 27402 raft_consensus.cc:2463] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:07.158365 27599 leader_election.cc:304] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:07.158449 27896 raft_consensus.cc:2798] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:07.158496 27896 raft_consensus.cc:491] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:07.158521 27896 raft_consensus.cc:3054] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.159143 27896 raft_consensus.cc:513] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.159281 27896 leader_election.cc:290] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Requested vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:07.159444 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60af3902e844f088d271b897635e987" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:07.159533 27402 raft_consensus.cc:3054] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.160199 27402 raft_consensus.cc:2463] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:07.160358 27599 leader_election.cc:304] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:07.160465 27896 raft_consensus.cc:2798] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:07.160528 27896 raft_consensus.cc:695] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [term 1 LEADER]: Becoming Leader. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Running, Role: LEADER
I20250112 21:06:07.160581 27896 consensus_queue.cc:237] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.160755 27137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60af3902e844f088d271b897635e987" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:07.160844 27137 raft_consensus.cc:3054] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.160744 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e60af3902e844f088d271b897635e987" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:07.161259 26993 catalog_manager.cc:5526] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69). New cstate: current_term: 1 leader_uuid: "9b470022ea814aa5bba1c6716c187a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:07.161528 27137 raft_consensus.cc:2463] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:07.239650 27889 consensus_queue.cc:1035] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.241216 27889 consensus_queue.cc:1035] T 377842729046491aa373c2c9244804c9 P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 21:06:07.294102 27982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:07.294253 27982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:07.295599 27982 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20250112 21:06:07.329210 27402 raft_consensus.cc:1270] T e60af3902e844f088d271b897635e987 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.329432 27137 raft_consensus.cc:1270] T e60af3902e844f088d271b897635e987 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.329447 27896 consensus_queue.cc:1035] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.330024 27896 consensus_queue.cc:1035] T e60af3902e844f088d271b897635e987 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.330252 27798 raft_consensus.cc:1270] T 2736e58da0e64362960fc8cc8612ecd2 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.330693 27270 raft_consensus.cc:1270] T 2736e58da0e64362960fc8cc8612ecd2 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.331240 27889 consensus_queue.cc:1035] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.332046 27889 consensus_queue.cc:1035] T 2736e58da0e64362960fc8cc8612ecd2 P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 20.9389 ms
time per row: 0.0104694 ms
I20250112 21:06:07.346259 26994 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:07.347533 27646 tablet_service.cc:1467] Processing CreateTablet for tablet 1795c2e5a6f64569a0f8baf08b44ff1e (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.347563 27118 tablet_service.cc:1467] Processing CreateTablet for tablet 1795c2e5a6f64569a0f8baf08b44ff1e (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.347673 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1795c2e5a6f64569a0f8baf08b44ff1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.347677 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1795c2e5a6f64569a0f8baf08b44ff1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.347708 27777 tablet_service.cc:1467] Processing CreateTablet for tablet 362bbb60e761455c906dfdb0ff0e6128 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.347805 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362bbb60e761455c906dfdb0ff0e6128. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.348601 28040 tablet_bootstrap.cc:492] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:07.349089 27118 tablet_service.cc:1467] Processing CreateTablet for tablet 87d3010fd7704bda93e4fc592bc4b3f9 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.349195 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d3010fd7704bda93e4fc592bc4b3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.349227 28040 tablet_bootstrap.cc:654] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.349365 28041 tablet_bootstrap.cc:492] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:07.349380 27381 tablet_service.cc:1467] Processing CreateTablet for tablet 1795c2e5a6f64569a0f8baf08b44ff1e (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.349476 27381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1795c2e5a6f64569a0f8baf08b44ff1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.349781 28041 tablet_bootstrap.cc:654] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.350118 27514 tablet_service.cc:1467] Processing CreateTablet for tablet 362bbb60e761455c906dfdb0ff0e6128 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.350222 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362bbb60e761455c906dfdb0ff0e6128. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.350423 28041 tablet_bootstrap.cc:492] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:07.350464 28041 ts_tablet_manager.cc:1397] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.350592 28041 raft_consensus.cc:357] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.350656 28041 raft_consensus.cc:383] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.350677 28041 raft_consensus.cc:738] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.350745 28041 consensus_queue.cc:260] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.350836 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 87d3010fd7704bda93e4fc592bc4b3f9 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.350878 28045 tablet_bootstrap.cc:492] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:07.350921 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d3010fd7704bda93e4fc592bc4b3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.351037 28041 ts_tablet_manager.cc:1428] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.351572 28045 tablet_bootstrap.cc:654] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.351647 28046 tablet_bootstrap.cc:492] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:07.351903 28042 tablet_bootstrap.cc:492] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:07.352072 28046 tablet_bootstrap.cc:654] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.352125 28045 tablet_bootstrap.cc:492] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:07.352108 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 362bbb60e761455c906dfdb0ff0e6128 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.352174 28045 ts_tablet_manager.cc:1397] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.352218 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362bbb60e761455c906dfdb0ff0e6128. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.352312 28045 raft_consensus.cc:357] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.352377 28045 raft_consensus.cc:383] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.352401 28045 raft_consensus.cc:738] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.352453 28045 consensus_queue.cc:260] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.352535 28045 ts_tablet_manager.cc:1428] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.352596 28045 tablet_bootstrap.cc:492] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:07.352603 28046 tablet_bootstrap.cc:492] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:07.352622 28042 tablet_bootstrap.cc:654] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.352641 28046 ts_tablet_manager.cc:1397] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.352798 28046 raft_consensus.cc:357] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.352872 28046 raft_consensus.cc:383] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.352897 28046 raft_consensus.cc:738] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.352941 28046 consensus_queue.cc:260] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.353021 28045 tablet_bootstrap.cc:654] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.353039 28046 ts_tablet_manager.cc:1428] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.353205 28042 tablet_bootstrap.cc:492] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:07.353247 28042 ts_tablet_manager.cc:1397] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.353394 28042 raft_consensus.cc:357] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.353454 28042 raft_consensus.cc:383] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.353479 28042 raft_consensus.cc:738] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.353538 28042 consensus_queue.cc:260] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.353634 28042 ts_tablet_manager.cc:1428] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:07.353833 28045 tablet_bootstrap.cc:492] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:07.353885 28045 ts_tablet_manager.cc:1397] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.353868 27646 tablet_service.cc:1467] Processing CreateTablet for tablet 87d3010fd7704bda93e4fc592bc4b3f9 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2066a7d010ea4b7aaab9fb55db81560f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.353964 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d3010fd7704bda93e4fc592bc4b3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.354020 28045 raft_consensus.cc:357] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.354084 28045 raft_consensus.cc:383] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.354110 28045 raft_consensus.cc:738] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.354156 28045 consensus_queue.cc:260] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.354238 28045 ts_tablet_manager.cc:1428] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.355193 28041 tablet_bootstrap.cc:492] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:07.355389 28051 tablet_bootstrap.cc:492] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:07.355619 28041 tablet_bootstrap.cc:654] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.355805 28051 tablet_bootstrap.cc:654] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.356221 28041 tablet_bootstrap.cc:492] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:07.356271 28041 ts_tablet_manager.cc:1397] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.356444 28041 raft_consensus.cc:357] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.356519 28041 raft_consensus.cc:383] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.356551 28041 raft_consensus.cc:738] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.356595 28041 consensus_queue.cc:260] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.356639 28051 tablet_bootstrap.cc:492] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:07.356674 28040 tablet_bootstrap.cc:492] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:07.356700 28051 ts_tablet_manager.cc:1397] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.356720 28040 ts_tablet_manager.cc:1397] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20250112 21:06:07.356729 28041 ts_tablet_manager.cc:1428] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:07.356866 28051 raft_consensus.cc:357] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.356936 28051 raft_consensus.cc:383] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.356958 28051 raft_consensus.cc:738] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.357018 28051 consensus_queue.cc:260] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.357031 28040 raft_consensus.cc:357] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.357102 28051 ts_tablet_manager.cc:1428] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.357102 28040 raft_consensus.cc:383] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.357137 28040 raft_consensus.cc:738] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.357179 28040 consensus_queue.cc:260] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.357266 28040 ts_tablet_manager.cc:1428] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.357323 28040 tablet_bootstrap.cc:492] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:07.357674 28040 tablet_bootstrap.cc:654] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.358218 28040 tablet_bootstrap.cc:492] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:07.358273 28040 ts_tablet_manager.cc:1397] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:07.358404 28040 raft_consensus.cc:357] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.358456 28040 raft_consensus.cc:383] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.358477 28040 raft_consensus.cc:738] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.358520 28040 consensus_queue.cc:260] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.358606 28040 ts_tablet_manager.cc:1428] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.420959 27900 raft_consensus.cc:491] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:07.421036 27900 raft_consensus.cc:513] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.421195 27900 leader_election.cc:290] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:07.421588 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362bbb60e761455c906dfdb0ff0e6128" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:07.421684 27534 raft_consensus.cc:2463] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:07.421837 27731 leader_election.cc:304] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.421938 27900 raft_consensus.cc:2798] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:07.421994 27900 raft_consensus.cc:491] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:07.422027 27900 raft_consensus.cc:3054] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.422770 27900 raft_consensus.cc:513] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.422901 27900 leader_election.cc:290] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Requested vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:07.423025 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362bbb60e761455c906dfdb0ff0e6128" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:07.423095 27534 raft_consensus.cc:3054] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.423583 27534 raft_consensus.cc:2463] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:07.423768 27731 leader_election.cc:304] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.423866 27900 raft_consensus.cc:2798] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:07.424010 27900 raft_consensus.cc:695] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Running, Role: LEADER
I20250112 21:06:07.424082 27900 consensus_queue.cc:237] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:07.424292 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362bbb60e761455c906dfdb0ff0e6128" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:07.424327 27269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362bbb60e761455c906dfdb0ff0e6128" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:07.424384 27270 raft_consensus.cc:2463] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:07.424757 26994 catalog_manager.cc:5526] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4aa042eb06ac407fa827c735ea972020 (127.26.82.70). New cstate: current_term: 1 leader_uuid: "4aa042eb06ac407fa827c735ea972020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:07.537477 28018 raft_consensus.cc:491] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:07.537556 28018 raft_consensus.cc:513] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.537690 28018 leader_election.cc:290] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:07.537910 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d3010fd7704bda93e4fc592bc4b3f9" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:07.537920 27137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d3010fd7704bda93e4fc592bc4b3f9" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:07.538002 27666 raft_consensus.cc:2463] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:07.538008 27137 raft_consensus.cc:2463] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:07.538163 27337 leader_election.cc:304] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:07.538268 28018 raft_consensus.cc:2798] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:07.538360 28018 raft_consensus.cc:491] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:07.538376 28018 raft_consensus.cc:3054] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.538972 28018 raft_consensus.cc:513] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.539083 28018 leader_election.cc:290] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Requested vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:07.539229 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d3010fd7704bda93e4fc592bc4b3f9" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:07.539255 27137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d3010fd7704bda93e4fc592bc4b3f9" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:07.539300 27666 raft_consensus.cc:3054] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.539319 27137 raft_consensus.cc:3054] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.539985 27666 raft_consensus.cc:2463] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:07.539985 27137 raft_consensus.cc:2463] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:07.540184 27337 leader_election.cc:304] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:07.540282 28018 raft_consensus.cc:2798] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:07.540439 28018 raft_consensus.cc:695] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 LEADER]: Becoming Leader. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Running, Role: LEADER
I20250112 21:06:07.540488 28018 consensus_queue.cc:237] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.541076 26994 catalog_manager.cc:5526] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: HEALTHY } } }
I20250112 21:06:07.541801 28018 raft_consensus.cc:491] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:07.541878 28018 raft_consensus.cc:513] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.541961 28018 leader_election.cc:290] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:07.542079 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1795c2e5a6f64569a0f8baf08b44ff1e" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:07.542097 27137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1795c2e5a6f64569a0f8baf08b44ff1e" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:07.542147 27666 raft_consensus.cc:2463] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:07.542151 27137 raft_consensus.cc:2463] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:07.542273 27337 leader_election.cc:304] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:07.542356 28018 raft_consensus.cc:2798] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:07.542392 28018 raft_consensus.cc:491] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:07.542416 28018 raft_consensus.cc:3054] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.542905 28018 raft_consensus.cc:513] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.543010 28018 leader_election.cc:290] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Requested vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:07.543128 27137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1795c2e5a6f64569a0f8baf08b44ff1e" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:07.543133 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1795c2e5a6f64569a0f8baf08b44ff1e" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:07.543187 27137 raft_consensus.cc:3054] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.543187 27666 raft_consensus.cc:3054] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.543670 27666 raft_consensus.cc:2463] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:07.543763 27137 raft_consensus.cc:2463] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:07.543864 27336 leader_election.cc:304] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:07.543959 28018 raft_consensus.cc:2798] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:07.544019 28018 raft_consensus.cc:695] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 LEADER]: Becoming Leader. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Running, Role: LEADER
I20250112 21:06:07.544070 28018 consensus_queue.cc:237] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:07.544588 26994 catalog_manager.cc:5526] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: HEALTHY } } }
W20250112 21:06:07.651638 28062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:07.651775 28062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:07.653052 28062 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20250112 21:06:07.682292 27534 raft_consensus.cc:1270] T 362bbb60e761455c906dfdb0ff0e6128 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.682430 27270 raft_consensus.cc:3054] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.682650 27137 raft_consensus.cc:1270] T 1795c2e5a6f64569a0f8baf08b44ff1e P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.683286 27270 raft_consensus.cc:1270] T 362bbb60e761455c906dfdb0ff0e6128 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.683418 28057 consensus_queue.cc:1035] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.683456 27666 raft_consensus.cc:1270] T 1795c2e5a6f64569a0f8baf08b44ff1e P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:07.683666 28018 consensus_queue.cc:1035] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.683840 27666 raft_consensus.cc:1270] T 87d3010fd7704bda93e4fc592bc4b3f9 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 21:06:07.683832 28018 consensus_queue.cc:1035] T 1795c2e5a6f64569a0f8baf08b44ff1e P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.683869 27137 raft_consensus.cc:1270] T 87d3010fd7704bda93e4fc592bc4b3f9 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 21:06:07.684101 28057 consensus_queue.cc:1035] T 362bbb60e761455c906dfdb0ff0e6128 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.685060 28095 consensus_queue.cc:1035] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:07.685410 28095 consensus_queue.cc:1035] T 87d3010fd7704bda93e4fc592bc4b3f9 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 23.0221 ms
time per row: 0.011511 ms
I20250112 21:06:07.699167 26991 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:07.700421 27777 tablet_service.cc:1467] Processing CreateTablet for tablet dfd0bc2a10f64bc399ed609d672cae17 (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.700556 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd0bc2a10f64bc399ed609d672cae17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.700533 27250 tablet_service.cc:1467] Processing CreateTablet for tablet dfd0bc2a10f64bc399ed609d672cae17 (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.700574 27118 tablet_service.cc:1467] Processing CreateTablet for tablet c72146655e9f44688b0a6071aa841d5a (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.700587 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 3abd309fa6ac4e5d8efd743e96f2998d (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.700649 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd0bc2a10f64bc399ed609d672cae17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.700683 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c72146655e9f44688b0a6071aa841d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.700729 27382 tablet_service.cc:1467] Processing CreateTablet for tablet c72146655e9f44688b0a6071aa841d5a (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.700831 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c72146655e9f44688b0a6071aa841d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.701813 28042 tablet_bootstrap.cc:492] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:07.701813 28040 tablet_bootstrap.cc:492] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:07.701951 28045 tablet_bootstrap.cc:492] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:07.701965 28051 tablet_bootstrap.cc:492] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:07.702188 28040 tablet_bootstrap.cc:654] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.702183 28042 tablet_bootstrap.cc:654] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.702286 28045 tablet_bootstrap.cc:654] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.702339 28051 tablet_bootstrap.cc:654] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.700428 27778 tablet_service.cc:1467] Processing CreateTablet for tablet 3abd309fa6ac4e5d8efd743e96f2998d (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.702550 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3abd309fa6ac4e5d8efd743e96f2998d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.702729 27646 tablet_service.cc:1467] Processing CreateTablet for tablet c72146655e9f44688b0a6071aa841d5a (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.702818 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c72146655e9f44688b0a6071aa841d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.702850 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3abd309fa6ac4e5d8efd743e96f2998d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.703071 28051 tablet_bootstrap.cc:492] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:07.703109 28051 ts_tablet_manager.cc:1397] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.703269 28051 raft_consensus.cc:357] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.703326 28051 raft_consensus.cc:383] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.703349 28051 raft_consensus.cc:738] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.703392 28051 consensus_queue.cc:260] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.703477 28051 ts_tablet_manager.cc:1428] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.703786 28040 tablet_bootstrap.cc:492] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:07.703833 28040 ts_tablet_manager.cc:1397] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:07.704001 28040 raft_consensus.cc:357] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:07.704070 28040 raft_consensus.cc:383] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.704042 27514 tablet_service.cc:1467] Processing CreateTablet for tablet dfd0bc2a10f64bc399ed609d672cae17 (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.704092 28040 raft_consensus.cc:738] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.704141 28040 consensus_queue.cc:260] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:07.704161 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfd0bc2a10f64bc399ed609d672cae17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.704246 28040 ts_tablet_manager.cc:1428] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.704381 28051 tablet_bootstrap.cc:492] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:07.704419 28045 tablet_bootstrap.cc:492] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:07.704458 28045 ts_tablet_manager.cc:1397] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:07.704583 28045 raft_consensus.cc:357] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:07.704633 28045 raft_consensus.cc:383] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.704651 28045 raft_consensus.cc:738] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.704715 28045 consensus_queue.cc:260] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:07.704814 28045 ts_tablet_manager.cc:1428] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:07.704895 28051 tablet_bootstrap.cc:654] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.705367 27513 tablet_service.cc:1467] Processing CreateTablet for tablet 3abd309fa6ac4e5d8efd743e96f2998d (DEFAULT_TABLE table=rebalance_test_table_3 [id=4e18ed0f19e943fa86a45cb4b6289a61]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:07.705443 28051 tablet_bootstrap.cc:492] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:07.705452 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3abd309fa6ac4e5d8efd743e96f2998d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:07.705483 28051 ts_tablet_manager.cc:1397] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:07.705624 28051 raft_consensus.cc:357] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.705695 28051 raft_consensus.cc:383] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.705713 28051 raft_consensus.cc:738] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.705754 28051 consensus_queue.cc:260] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.705917 28051 ts_tablet_manager.cc:1428] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:07.706215 28042 tablet_bootstrap.cc:492] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:07.706256 28042 ts_tablet_manager.cc:1397] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250112 21:06:07.706409 28042 raft_consensus.cc:357] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.706473 28042 raft_consensus.cc:383] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.706496 28042 raft_consensus.cc:738] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.706548 28041 tablet_bootstrap.cc:492] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:07.706552 28042 consensus_queue.cc:260] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.706658 28042 ts_tablet_manager.cc:1428] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.706717 28042 tablet_bootstrap.cc:492] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:07.707142 28041 tablet_bootstrap.cc:654] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.707141 28042 tablet_bootstrap.cc:654] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.707623 28046 tablet_bootstrap.cc:492] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:07.707748 28041 tablet_bootstrap.cc:492] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:07.707792 28041 ts_tablet_manager.cc:1397] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:07.707924 28041 raft_consensus.cc:357] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:07.707993 28041 raft_consensus.cc:383] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.708030 28041 raft_consensus.cc:738] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.708060 28046 tablet_bootstrap.cc:654] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.708083 28041 consensus_queue.cc:260] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:07.708202 28041 ts_tablet_manager.cc:1428] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.708391 28042 tablet_bootstrap.cc:492] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:07.708431 28042 ts_tablet_manager.cc:1397] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:07.708552 28042 raft_consensus.cc:357] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.708614 28042 raft_consensus.cc:383] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.708637 28042 raft_consensus.cc:738] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.708705 28042 consensus_queue.cc:260] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.708797 28042 ts_tablet_manager.cc:1428] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.708884 28046 tablet_bootstrap.cc:492] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:07.708923 28046 ts_tablet_manager.cc:1397] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.709023 28046 raft_consensus.cc:357] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.709059 28046 raft_consensus.cc:383] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.709072 28046 raft_consensus.cc:738] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.709103 28046 consensus_queue.cc:260] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.709167 28046 ts_tablet_manager.cc:1428] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.709208 28046 tablet_bootstrap.cc:492] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:07.709555 28046 tablet_bootstrap.cc:654] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:07.710206 28046 tablet_bootstrap.cc:492] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:07.710250 28046 ts_tablet_manager.cc:1397] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:07.710402 28046 raft_consensus.cc:357] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.710470 28046 raft_consensus.cc:383] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:07.710495 28046 raft_consensus.cc:738] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:07.710534 28046 consensus_queue.cc:260] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.710621 28046 ts_tablet_manager.cc:1428] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:07.935173 27884 raft_consensus.cc:491] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:07.935253 27884 raft_consensus.cc:513] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.935546 27884 leader_election.cc:290] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:07.938148 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd0bc2a10f64bc399ed609d672cae17" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:07.938169 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd0bc2a10f64bc399ed609d672cae17" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:07.938247 27534 raft_consensus.cc:2463] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:07.938247 27797 raft_consensus.cc:2463] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:07.938405 27203 leader_election.cc:304] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.938521 27884 raft_consensus.cc:2798] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:07.938562 27884 raft_consensus.cc:491] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:07.938584 27884 raft_consensus.cc:3054] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.939144 27884 raft_consensus.cc:513] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.939253 27884 leader_election.cc:290] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Requested vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:07.939380 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd0bc2a10f64bc399ed609d672cae17" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:07.939411 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd0bc2a10f64bc399ed609d672cae17" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:07.939445 27797 raft_consensus.cc:3054] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.939477 27534 raft_consensus.cc:3054] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.939978 27534 raft_consensus.cc:2463] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:07.939990 27797 raft_consensus.cc:2463] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:07.940125 27203 leader_election.cc:304] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.940227 27884 raft_consensus.cc:2798] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:07.940315 27884 raft_consensus.cc:695] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [term 1 LEADER]: Becoming Leader. State: Replica: c60c223c1269463181610419f08e11fc, State: Running, Role: LEADER
I20250112 21:06:07.940390 27884 consensus_queue.cc:237] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.941030 26991 catalog_manager.cc:5526] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c223c1269463181610419f08e11fc (127.26.82.66). New cstate: current_term: 1 leader_uuid: "c60c223c1269463181610419f08e11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: HEALTHY } } }
I20250112 21:06:07.971441 28017 raft_consensus.cc:491] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:07.971514 28017 raft_consensus.cc:513] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.971657 28017 leader_election.cc:290] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:07.971864 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abd309fa6ac4e5d8efd743e96f2998d" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:07.971948 27270 raft_consensus.cc:2463] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:07.972102 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abd309fa6ac4e5d8efd743e96f2998d" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:07.972172 27797 raft_consensus.cc:2463] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:07.972297 27468 leader_election.cc:304] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.972388 28017 raft_consensus.cc:2798] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:07.972422 28017 raft_consensus.cc:491] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:07.972451 28017 raft_consensus.cc:3054] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.973187 28017 raft_consensus.cc:513] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.973304 28017 leader_election.cc:290] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Requested vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:07.973517 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abd309fa6ac4e5d8efd743e96f2998d" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:07.973589 27270 raft_consensus.cc:3054] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.973868 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abd309fa6ac4e5d8efd743e96f2998d" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:07.973923 27797 raft_consensus.cc:3054] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:07.974277 27270 raft_consensus.cc:2463] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:07.974551 27797 raft_consensus.cc:2463] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:07.974732 27468 leader_election.cc:304] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:07.974843 28017 raft_consensus.cc:2798] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:07.974962 28017 raft_consensus.cc:695] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 LEADER]: Becoming Leader. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Running, Role: LEADER
I20250112 21:06:07.975010 28017 consensus_queue.cc:237] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:07.975484 26991 catalog_manager.cc:5526] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 reported cstate change: term changed from 0 to 1, leader changed from <none> to efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68). New cstate: current_term: 1 leader_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:08.035511 28027 raft_consensus.cc:491] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.035594 28027 raft_consensus.cc:513] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.035872 28027 leader_election.cc:290] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:08.038416 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72146655e9f44688b0a6071aa841d5a" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:08.038437 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72146655e9f44688b0a6071aa841d5a" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:08.038491 27402 raft_consensus.cc:2463] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:08.038506 27665 raft_consensus.cc:2463] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:08.038626 27071 leader_election.cc:304] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:08.038734 28027 raft_consensus.cc:2798] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.038789 28027 raft_consensus.cc:491] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.038811 28027 raft_consensus.cc:3054] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.039378 28027 raft_consensus.cc:513] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.039490 28027 leader_election.cc:290] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Requested vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:08.039685 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72146655e9f44688b0a6071aa841d5a" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:08.039773 27665 raft_consensus.cc:3054] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.040344 27665 raft_consensus.cc:2463] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:08.040446 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72146655e9f44688b0a6071aa841d5a" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:08.040518 27402 raft_consensus.cc:3054] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.040930 27072 leader_election.cc:304] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:08.041061 28027 raft_consensus.cc:2798] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.041127 27402 raft_consensus.cc:2463] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:08.041221 28027 raft_consensus.cc:695] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [term 1 LEADER]: Becoming Leader. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Running, Role: LEADER
I20250112 21:06:08.041301 28027 consensus_queue.cc:237] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.041877 26991 catalog_manager.cc:5526] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "90f636b130e64c04a7f6cdad4839ce33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } }
W20250112 21:06:08.132020 28126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:08.132212 28126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:08.133539 28126 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20250112 21:06:08.163618 27797 raft_consensus.cc:1270] T 3abd309fa6ac4e5d8efd743e96f2998d P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.164296 27270 raft_consensus.cc:1270] T 3abd309fa6ac4e5d8efd743e96f2998d P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.164597 28119 consensus_queue.cc:1035] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.164750 28017 consensus_queue.cc:1035] T 3abd309fa6ac4e5d8efd743e96f2998d P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.165234 27534 raft_consensus.cc:1270] T dfd0bc2a10f64bc399ed609d672cae17 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.165374 27797 raft_consensus.cc:1270] T dfd0bc2a10f64bc399ed609d672cae17 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.165592 27884 consensus_queue.cc:1035] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.167186 27665 raft_consensus.cc:1270] T c72146655e9f44688b0a6071aa841d5a P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.167788 27884 consensus_queue.cc:1035] T dfd0bc2a10f64bc399ed609d672cae17 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.167886 28027 consensus_queue.cc:1035] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.168296 27402 raft_consensus.cc:1270] T c72146655e9f44688b0a6071aa841d5a P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.169344 28027 consensus_queue.cc:1035] T c72146655e9f44688b0a6071aa841d5a P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 26.9778 ms
time per row: 0.0134889 ms
I20250112 21:06:08.183280 26994 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:08.184613 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 73a992ed65bc42c6b7c81e1ce36ce39d (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.184772 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a992ed65bc42c6b7c81e1ce36ce39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.184762 27249 tablet_service.cc:1467] Processing CreateTablet for tablet b2cfdb7868d6468f9bd3e0c6394278ba (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.184782 27513 tablet_service.cc:1467] Processing CreateTablet for tablet b2cfdb7868d6468f9bd3e0c6394278ba (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.184878 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2cfdb7868d6468f9bd3e0c6394278ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.184882 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2cfdb7868d6468f9bd3e0c6394278ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.185842 28045 tablet_bootstrap.cc:492] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:08.186033 28051 tablet_bootstrap.cc:492] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:08.186035 28046 tablet_bootstrap.cc:492] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:08.186244 28045 tablet_bootstrap.cc:654] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.186380 27646 tablet_service.cc:1467] Processing CreateTablet for tablet 73a992ed65bc42c6b7c81e1ce36ce39d (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.186447 28046 tablet_bootstrap.cc:654] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.186486 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a992ed65bc42c6b7c81e1ce36ce39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.186625 28051 tablet_bootstrap.cc:654] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.186810 27778 tablet_service.cc:1467] Processing CreateTablet for tablet b2cfdb7868d6468f9bd3e0c6394278ba (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.186935 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2cfdb7868d6468f9bd3e0c6394278ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.187032 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 536162d5101c4d3685df680155707da7 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.187108 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536162d5101c4d3685df680155707da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.187690 28041 tablet_bootstrap.cc:492] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:08.187716 27645 tablet_service.cc:1467] Processing CreateTablet for tablet 536162d5101c4d3685df680155707da7 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.187798 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536162d5101c4d3685df680155707da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.188040 28041 tablet_bootstrap.cc:654] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.188196 28051 tablet_bootstrap.cc:492] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:08.188213 28042 tablet_bootstrap.cc:492] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:08.188246 28051 ts_tablet_manager.cc:1397] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:08.188395 28051 raft_consensus.cc:357] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.188457 28051 raft_consensus.cc:383] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.188482 28051 raft_consensus.cc:738] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.188529 28051 consensus_queue.cc:260] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.188566 28042 tablet_bootstrap.cc:654] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.188632 28051 ts_tablet_manager.cc:1428] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.189292 28042 tablet_bootstrap.cc:492] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:08.189308 27118 tablet_service.cc:1467] Processing CreateTablet for tablet 73a992ed65bc42c6b7c81e1ce36ce39d (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.189347 28042 ts_tablet_manager.cc:1397] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:08.189419 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a992ed65bc42c6b7c81e1ce36ce39d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.189451 28045 tablet_bootstrap.cc:492] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:08.189489 28045 ts_tablet_manager.cc:1397] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250112 21:06:08.189505 28042 raft_consensus.cc:357] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.189572 28042 raft_consensus.cc:383] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.189590 28042 raft_consensus.cc:738] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.189642 28042 consensus_queue.cc:260] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.189630 28045 raft_consensus.cc:357] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.189688 28045 raft_consensus.cc:383] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.189709 28045 raft_consensus.cc:738] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.189733 28042 ts_tablet_manager.cc:1428] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.189749 28045 consensus_queue.cc:260] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.189826 27117 tablet_service.cc:1467] Processing CreateTablet for tablet 536162d5101c4d3685df680155707da7 (DEFAULT_TABLE table=rebalance_test_table_4 [id=d4c444719b704d82bb5584ccf610401c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.189851 28045 ts_tablet_manager.cc:1428] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:08.189921 28045 tablet_bootstrap.cc:492] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:08.189922 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536162d5101c4d3685df680155707da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.190290 28045 tablet_bootstrap.cc:654] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.190701 28040 tablet_bootstrap.cc:492] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:08.190745 28046 tablet_bootstrap.cc:492] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:08.190791 28046 ts_tablet_manager.cc:1397] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250112 21:06:08.190923 28046 raft_consensus.cc:357] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.190965 28046 raft_consensus.cc:383] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.190979 28046 raft_consensus.cc:738] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.191012 28046 consensus_queue.cc:260] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.191035 28040 tablet_bootstrap.cc:654] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.191107 28046 ts_tablet_manager.cc:1428] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:08.191459 28041 tablet_bootstrap.cc:492] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:08.191504 28041 ts_tablet_manager.cc:1397] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250112 21:06:08.191618 28041 raft_consensus.cc:357] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.191677 28041 raft_consensus.cc:383] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.191699 28041 raft_consensus.cc:738] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.191757 28041 consensus_queue.cc:260] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.191846 28041 ts_tablet_manager.cc:1428] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.191905 28041 tablet_bootstrap.cc:492] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:08.192307 28041 tablet_bootstrap.cc:654] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.192308 28045 tablet_bootstrap.cc:492] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:08.192359 28045 ts_tablet_manager.cc:1397] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:08.192390 28040 tablet_bootstrap.cc:492] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:08.192438 28040 ts_tablet_manager.cc:1397] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:08.192481 28045 raft_consensus.cc:357] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.192546 28045 raft_consensus.cc:383] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.192577 28045 raft_consensus.cc:738] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.192555 28040 raft_consensus.cc:357] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.192624 28040 raft_consensus.cc:383] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.192678 28040 raft_consensus.cc:738] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.192730 28040 consensus_queue.cc:260] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.192622 28045 consensus_queue.cc:260] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.192826 28045 ts_tablet_manager.cc:1428] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:08.192826 28040 ts_tablet_manager.cc:1428] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.192886 28040 tablet_bootstrap.cc:492] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:08.193251 28040 tablet_bootstrap.cc:654] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.193315 28041 tablet_bootstrap.cc:492] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:08.193359 28041 ts_tablet_manager.cc:1397] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:08.193483 28041 raft_consensus.cc:357] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.193529 28041 raft_consensus.cc:383] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.193542 28041 raft_consensus.cc:738] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.193578 28041 consensus_queue.cc:260] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.193660 28041 ts_tablet_manager.cc:1428] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.194025 28040 tablet_bootstrap.cc:492] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:08.194079 28040 ts_tablet_manager.cc:1397] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:08.194208 28040 raft_consensus.cc:357] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.194269 28040 raft_consensus.cc:383] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.194290 28040 raft_consensus.cc:738] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.194329 28040 consensus_queue.cc:260] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.194399 28040 ts_tablet_manager.cc:1428] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.206722 28110 raft_consensus.cc:491] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.206785 28110 raft_consensus.cc:513] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.206919 28110 leader_election.cc:290] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:08.207051 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cfdb7868d6468f9bd3e0c6394278ba" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:08.207087 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cfdb7868d6468f9bd3e0c6394278ba" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:08.207134 27534 raft_consensus.cc:2463] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:08.207149 27797 raft_consensus.cc:2463] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:08.207291 27204 leader_election.cc:304] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:08.207389 28110 raft_consensus.cc:2798] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.207433 28110 raft_consensus.cc:491] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.207448 28110 raft_consensus.cc:3054] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.208012 28110 raft_consensus.cc:513] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.208134 28110 leader_election.cc:290] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:08.208267 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cfdb7868d6468f9bd3e0c6394278ba" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:08.208328 27797 raft_consensus.cc:3054] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.208851 27797 raft_consensus.cc:2463] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:08.208948 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cfdb7868d6468f9bd3e0c6394278ba" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:08.208993 27204 leader_election.cc:304] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:08.209005 27534 raft_consensus.cc:3054] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.209057 28110 raft_consensus.cc:2798] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.209106 28110 raft_consensus.cc:695] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [term 1 LEADER]: Becoming Leader. State: Replica: c60c223c1269463181610419f08e11fc, State: Running, Role: LEADER
I20250112 21:06:08.209153 28110 consensus_queue.cc:237] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.209571 26989 catalog_manager.cc:5526] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c223c1269463181610419f08e11fc (127.26.82.66). New cstate: current_term: 1 leader_uuid: "c60c223c1269463181610419f08e11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:08.209671 27534 raft_consensus.cc:2463] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:08.215541 28166 raft_consensus.cc:491] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.215603 28166 raft_consensus.cc:513] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.215731 28166 leader_election.cc:290] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:08.215924 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536162d5101c4d3685df680155707da7" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:08.215935 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536162d5101c4d3685df680155707da7" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:08.215989 27665 raft_consensus.cc:2463] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:08.216004 27402 raft_consensus.cc:2463] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:08.216152 27072 leader_election.cc:304] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:08.216249 28166 raft_consensus.cc:2798] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.216290 28166 raft_consensus.cc:491] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.216305 28166 raft_consensus.cc:3054] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.216915 28166 raft_consensus.cc:513] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.217052 28166 leader_election.cc:290] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Requested vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:08.217167 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536162d5101c4d3685df680155707da7" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:08.217178 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536162d5101c4d3685df680155707da7" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:08.217224 27665 raft_consensus.cc:3054] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.217237 27402 raft_consensus.cc:3054] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.217743 27402 raft_consensus.cc:2463] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:08.217902 27071 leader_election.cc:304] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:08.217996 28166 raft_consensus.cc:2798] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.218066 28166 raft_consensus.cc:695] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 LEADER]: Becoming Leader. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Running, Role: LEADER
I20250112 21:06:08.218080 27665 raft_consensus.cc:2463] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:08.218127 28166 consensus_queue.cc:237] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.218583 26989 catalog_manager.cc:5526] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "90f636b130e64c04a7f6cdad4839ce33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: HEALTHY } } }
I20250112 21:06:08.335418 27876 raft_consensus.cc:491] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.335505 27876 raft_consensus.cc:513] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.335659 27876 leader_election.cc:290] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:08.335836 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a992ed65bc42c6b7c81e1ce36ce39d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:08.335834 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a992ed65bc42c6b7c81e1ce36ce39d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:08.335916 27138 raft_consensus.cc:2463] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:08.335924 27402 raft_consensus.cc:2463] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:08.336076 27601 leader_election.cc:304] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:08.336184 27876 raft_consensus.cc:2798] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.336230 27876 raft_consensus.cc:491] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.336243 27876 raft_consensus.cc:3054] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.336889 27876 raft_consensus.cc:513] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.337005 27876 leader_election.cc:290] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Requested vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:08.337136 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a992ed65bc42c6b7c81e1ce36ce39d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:08.337170 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a992ed65bc42c6b7c81e1ce36ce39d" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:08.337217 27402 raft_consensus.cc:3054] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.337229 27138 raft_consensus.cc:3054] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.337888 27402 raft_consensus.cc:2463] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:08.337886 27138 raft_consensus.cc:2463] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:08.338064 27599 leader_election.cc:304] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:08.338166 27876 raft_consensus.cc:2798] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.338261 27876 raft_consensus.cc:695] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [term 1 LEADER]: Becoming Leader. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Running, Role: LEADER
I20250112 21:06:08.338310 27876 consensus_queue.cc:237] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.338817 26989 catalog_manager.cc:5526] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69). New cstate: current_term: 1 leader_uuid: "9b470022ea814aa5bba1c6716c187a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } }
W20250112 21:06:08.443194 28183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:08.443356 28183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:08.444530 28183 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20250112 21:06:08.473754 27534 raft_consensus.cc:1270] T b2cfdb7868d6468f9bd3e0c6394278ba P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.474100 27797 raft_consensus.cc:1270] T b2cfdb7868d6468f9bd3e0c6394278ba P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.474061 27884 consensus_queue.cc:1035] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.474285 27402 raft_consensus.cc:1270] T 536162d5101c4d3685df680155707da7 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.474308 27884 consensus_queue.cc:1035] T b2cfdb7868d6468f9bd3e0c6394278ba P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.474516 28027 consensus_queue.cc:1035] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.474980 27665 raft_consensus.cc:1270] T 536162d5101c4d3685df680155707da7 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.475299 28027 consensus_queue.cc:1035] T 536162d5101c4d3685df680155707da7 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.480146 27401 raft_consensus.cc:1270] T 73a992ed65bc42c6b7c81e1ce36ce39d P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.480306 27138 raft_consensus.cc:1270] T 73a992ed65bc42c6b7c81e1ce36ce39d P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.480718 28181 consensus_queue.cc:1035] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.480875 28181 consensus_queue.cc:1035] T 73a992ed65bc42c6b7c81e1ce36ce39d P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 27.85 ms
time per row: 0.013925 ms
I20250112 21:06:08.496964 26989 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:08.498500 27778 tablet_service.cc:1467] Processing CreateTablet for tablet 1108d4d8d0154ebe86fa0c119480f389 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.498594 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 1108d4d8d0154ebe86fa0c119480f389 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.498589 27513 tablet_service.cc:1467] Processing CreateTablet for tablet 1108d4d8d0154ebe86fa0c119480f389 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.498646 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1108d4d8d0154ebe86fa0c119480f389. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.498589 27514 tablet_service.cc:1467] Processing CreateTablet for tablet b8c15dc68d6d453a8c8f353f91ac8b5b (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.498692 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1108d4d8d0154ebe86fa0c119480f389. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.498744 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c15dc68d6d453a8c8f353f91ac8b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.498525 27777 tablet_service.cc:1467] Processing CreateTablet for tablet b8c15dc68d6d453a8c8f353f91ac8b5b (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.498853 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c15dc68d6d453a8c8f353f91ac8b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.499863 27250 tablet_service.cc:1467] Processing CreateTablet for tablet b8c15dc68d6d453a8c8f353f91ac8b5b (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.499864 27382 tablet_service.cc:1467] Processing CreateTablet for tablet de37ad95d2b74c5db78c175cf4ff8632 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.499951 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c15dc68d6d453a8c8f353f91ac8b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.499958 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de37ad95d2b74c5db78c175cf4ff8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.499974 28042 tablet_bootstrap.cc:492] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:08.499965 28046 tablet_bootstrap.cc:492] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:08.500355 28042 tablet_bootstrap.cc:654] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.500391 28046 tablet_bootstrap.cc:654] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.501034 28051 tablet_bootstrap.cc:492] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:08.501046 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1108d4d8d0154ebe86fa0c119480f389. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.501140 28045 tablet_bootstrap.cc:492] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:08.501360 28051 tablet_bootstrap.cc:654] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.501502 28045 tablet_bootstrap.cc:654] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.501958 28051 tablet_bootstrap.cc:492] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:08.502002 28051 ts_tablet_manager.cc:1397] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:08.502068 28045 tablet_bootstrap.cc:492] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:08.502105 28045 ts_tablet_manager.cc:1397] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:08.502137 28051 raft_consensus.cc:357] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.502200 28051 raft_consensus.cc:383] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.502218 28051 raft_consensus.cc:738] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.502235 28045 raft_consensus.cc:357] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.502264 28051 consensus_queue.cc:260] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.502295 28045 raft_consensus.cc:383] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.502317 28045 raft_consensus.cc:738] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.502355 28051 ts_tablet_manager.cc:1428] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:08.502352 28045 consensus_queue.cc:260] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.502408 28051 tablet_bootstrap.cc:492] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:08.502457 28045 ts_tablet_manager.cc:1428] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.502771 28042 tablet_bootstrap.cc:492] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:08.502792 28051 tablet_bootstrap.cc:654] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.502825 28042 ts_tablet_manager.cc:1397] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:08.502966 28042 raft_consensus.cc:357] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.503019 28042 raft_consensus.cc:383] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.503033 28042 raft_consensus.cc:738] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.503072 28042 consensus_queue.cc:260] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.503187 28042 ts_tablet_manager.cc:1428] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.503186 27645 tablet_service.cc:1467] Processing CreateTablet for tablet de37ad95d2b74c5db78c175cf4ff8632 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.503252 28042 tablet_bootstrap.cc:492] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:08.503284 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de37ad95d2b74c5db78c175cf4ff8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.503429 27117 tablet_service.cc:1467] Processing CreateTablet for tablet de37ad95d2b74c5db78c175cf4ff8632 (DEFAULT_TABLE table=rebalance_test_table_5 [id=d45ed0a3acf44f8dbc685886ff16b8ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.503520 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de37ad95d2b74c5db78c175cf4ff8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.503621 28042 tablet_bootstrap.cc:654] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.504276 28042 tablet_bootstrap.cc:492] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:08.504308 28042 ts_tablet_manager.cc:1397] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:08.504429 28042 raft_consensus.cc:357] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.504484 28042 raft_consensus.cc:383] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.504504 28042 raft_consensus.cc:738] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.504520 28041 tablet_bootstrap.cc:492] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:08.504541 28042 consensus_queue.cc:260] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.504616 28042 ts_tablet_manager.cc:1428] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.504859 28041 tablet_bootstrap.cc:654] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.504938 28040 tablet_bootstrap.cc:492] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:08.505440 28041 tablet_bootstrap.cc:492] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:08.505478 28040 tablet_bootstrap.cc:654] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.505493 28041 ts_tablet_manager.cc:1397] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:08.505611 28041 raft_consensus.cc:357] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.505673 28041 raft_consensus.cc:383] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.505692 28041 raft_consensus.cc:738] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.505728 28041 consensus_queue.cc:260] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.505822 28041 ts_tablet_manager.cc:1428] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.506305 28051 tablet_bootstrap.cc:492] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:08.506358 28051 ts_tablet_manager.cc:1397] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250112 21:06:08.506548 28051 raft_consensus.cc:357] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.506595 28051 raft_consensus.cc:383] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.506609 28051 raft_consensus.cc:738] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.506651 28051 consensus_queue.cc:260] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.506740 28051 ts_tablet_manager.cc:1428] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.506974 28040 tablet_bootstrap.cc:492] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:08.507018 28040 ts_tablet_manager.cc:1397] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:08.507124 28040 raft_consensus.cc:357] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.507181 28040 raft_consensus.cc:383] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.507212 28040 raft_consensus.cc:738] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.507256 28040 consensus_queue.cc:260] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.507354 28040 ts_tablet_manager.cc:1428] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.507752 28046 tablet_bootstrap.cc:492] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:08.507802 28046 ts_tablet_manager.cc:1397] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250112 21:06:08.507956 28046 raft_consensus.cc:357] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.508054 28046 raft_consensus.cc:383] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.508075 28046 raft_consensus.cc:738] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.508124 28046 consensus_queue.cc:260] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.508231 28046 ts_tablet_manager.cc:1428] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:08.508292 28046 tablet_bootstrap.cc:492] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:08.508738 28046 tablet_bootstrap.cc:654] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.509243 28046 tablet_bootstrap.cc:492] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:08.509285 28046 ts_tablet_manager.cc:1397] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:08.509433 28046 raft_consensus.cc:357] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.509503 28046 raft_consensus.cc:383] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.509522 28046 raft_consensus.cc:738] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.509564 28046 consensus_queue.cc:260] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.509676 28046 ts_tablet_manager.cc:1428] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:08.531275 28110 raft_consensus.cc:491] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.531337 28110 raft_consensus.cc:513] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.531481 28110 leader_election.cc:290] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:08.531602 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1108d4d8d0154ebe86fa0c119480f389" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:08.531647 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1108d4d8d0154ebe86fa0c119480f389" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:08.531690 27534 raft_consensus.cc:2463] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:08.531708 27797 raft_consensus.cc:2463] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:08.531821 27203 leader_election.cc:304] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:08.531908 28110 raft_consensus.cc:2798] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.531950 28110 raft_consensus.cc:491] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.531971 28110 raft_consensus.cc:3054] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.532614 28110 raft_consensus.cc:513] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.532759 28110 leader_election.cc:290] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:08.532886 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1108d4d8d0154ebe86fa0c119480f389" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:08.532943 27797 raft_consensus.cc:3054] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.532944 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1108d4d8d0154ebe86fa0c119480f389" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:08.533023 27534 raft_consensus.cc:3054] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.533468 27797 raft_consensus.cc:2463] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:08.533567 27534 raft_consensus.cc:2463] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:08.533634 27204 leader_election.cc:304] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:08.533722 28110 raft_consensus.cc:2798] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.533768 28110 raft_consensus.cc:695] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [term 1 LEADER]: Becoming Leader. State: Replica: c60c223c1269463181610419f08e11fc, State: Running, Role: LEADER
I20250112 21:06:08.533823 28110 consensus_queue.cc:237] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.534355 26996 catalog_manager.cc:5526] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c223c1269463181610419f08e11fc (127.26.82.66). New cstate: current_term: 1 leader_uuid: "c60c223c1269463181610419f08e11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:08.579504 28095 raft_consensus.cc:491] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.579586 28095 raft_consensus.cc:513] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.579730 28095 leader_election.cc:290] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:08.579946 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37ad95d2b74c5db78c175cf4ff8632" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:08.579943 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37ad95d2b74c5db78c175cf4ff8632" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:08.580024 27138 raft_consensus.cc:2463] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:08.580027 27665 raft_consensus.cc:2463] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:08.580192 27337 leader_election.cc:304] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:08.580300 28095 raft_consensus.cc:2798] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.580343 28095 raft_consensus.cc:491] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.580358 28095 raft_consensus.cc:3054] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.580924 28095 raft_consensus.cc:513] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.581034 28095 leader_election.cc:290] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Requested vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:08.581202 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37ad95d2b74c5db78c175cf4ff8632" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:08.581279 27665 raft_consensus.cc:3054] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.581264 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37ad95d2b74c5db78c175cf4ff8632" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:08.581339 27138 raft_consensus.cc:3054] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.581948 27665 raft_consensus.cc:2463] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:08.582003 27138 raft_consensus.cc:2463] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:08.582130 27336 leader_election.cc:304] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:08.582230 28095 raft_consensus.cc:2798] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.582330 28095 raft_consensus.cc:695] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 LEADER]: Becoming Leader. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Running, Role: LEADER
I20250112 21:06:08.582373 28095 consensus_queue.cc:237] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.582886 26996 catalog_manager.cc:5526] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:08.738874 28110 raft_consensus.cc:491] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.738950 28110 raft_consensus.cc:513] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.739109 28110 leader_election.cc:290] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:08.739253 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c15dc68d6d453a8c8f353f91ac8b5b" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:08.739277 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c15dc68d6d453a8c8f353f91ac8b5b" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:08.739409 27534 raft_consensus.cc:2463] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:08.739414 27797 raft_consensus.cc:2463] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:08.739552 27203 leader_election.cc:304] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:08.739619 28110 raft_consensus.cc:2798] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.739657 28110 raft_consensus.cc:491] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.739674 28110 raft_consensus.cc:3054] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.740217 28110 raft_consensus.cc:513] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.740314 28110 leader_election.cc:290] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Requested vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:08.740428 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c15dc68d6d453a8c8f353f91ac8b5b" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:08.740496 27797 raft_consensus.cc:3054] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.740476 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c15dc68d6d453a8c8f353f91ac8b5b" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:08.740523 27534 raft_consensus.cc:3054] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.741232 27797 raft_consensus.cc:2463] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:08.741232 27534 raft_consensus.cc:2463] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:08.741389 27204 leader_election.cc:304] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:08.741488 28110 raft_consensus.cc:2798] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.741549 28110 raft_consensus.cc:695] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [term 1 LEADER]: Becoming Leader. State: Replica: c60c223c1269463181610419f08e11fc, State: Running, Role: LEADER
I20250112 21:06:08.741605 28110 consensus_queue.cc:237] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:08.742162 26996 catalog_manager.cc:5526] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c223c1269463181610419f08e11fc (127.26.82.66). New cstate: current_term: 1 leader_uuid: "c60c223c1269463181610419f08e11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: HEALTHY } } }
W20250112 21:06:08.852923 28236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:08.853063 28236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:08.854194 28236 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20250112 21:06:08.882619 27797 raft_consensus.cc:1270] T 1108d4d8d0154ebe86fa0c119480f389 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.882673 27534 raft_consensus.cc:1270] T 1108d4d8d0154ebe86fa0c119480f389 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.882736 27798 raft_consensus.cc:1270] T b8c15dc68d6d453a8c8f353f91ac8b5b P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.882964 27884 consensus_queue.cc:1035] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.883101 27884 consensus_queue.cc:1035] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.882683 27533 raft_consensus.cc:1270] T b8c15dc68d6d453a8c8f353f91ac8b5b P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.884153 27138 raft_consensus.cc:1270] T de37ad95d2b74c5db78c175cf4ff8632 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.884325 27665 raft_consensus.cc:1270] T de37ad95d2b74c5db78c175cf4ff8632 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:08.884387 28095 consensus_queue.cc:1035] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.884842 28110 consensus_queue.cc:1035] T 1108d4d8d0154ebe86fa0c119480f389 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.884953 28110 consensus_queue.cc:1035] T b8c15dc68d6d453a8c8f353f91ac8b5b P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:08.885329 28095 consensus_queue.cc:1035] T de37ad95d2b74c5db78c175cf4ff8632 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 24.8101 ms
time per row: 0.012405 ms
I20250112 21:06:08.901866 26989 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:08.903124 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 38f1198a8bb24c1aa3094bb61d35d39c (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.903117 27117 tablet_service.cc:1467] Processing CreateTablet for tablet 38f1198a8bb24c1aa3094bb61d35d39c (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.903234 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f1198a8bb24c1aa3094bb61d35d39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.903235 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f1198a8bb24c1aa3094bb61d35d39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.903204 27513 tablet_service.cc:1467] Processing CreateTablet for tablet 68ba57a7dac345c29943b93e65c035bb (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.903324 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ba57a7dac345c29943b93e65c035bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.903337 27646 tablet_service.cc:1467] Processing CreateTablet for tablet dbdd9ee5599c41a39985f0338da8dc38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.903445 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdd9ee5599c41a39985f0338da8dc38. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.904379 28040 tablet_bootstrap.cc:492] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:08.904412 27645 tablet_service.cc:1467] Processing CreateTablet for tablet 38f1198a8bb24c1aa3094bb61d35d39c (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.904520 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f1198a8bb24c1aa3094bb61d35d39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.904693 28046 tablet_bootstrap.cc:492] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:08.904808 28045 tablet_bootstrap.cc:492] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:08.904829 28040 tablet_bootstrap.cc:654] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.905217 28045 tablet_bootstrap.cc:654] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.905364 28046 tablet_bootstrap.cc:654] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.905402 27117 tablet_service.cc:1467] Processing CreateTablet for tablet dbdd9ee5599c41a39985f0338da8dc38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.905485 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdd9ee5599c41a39985f0338da8dc38. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.905596 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 68ba57a7dac345c29943b93e65c035bb (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.905658 28041 tablet_bootstrap.cc:492] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:08.905699 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ba57a7dac345c29943b93e65c035bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.906065 28041 tablet_bootstrap.cc:654] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.906399 27777 tablet_service.cc:1467] Processing CreateTablet for tablet 68ba57a7dac345c29943b93e65c035bb (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.906513 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ba57a7dac345c29943b93e65c035bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.906571 28040 tablet_bootstrap.cc:492] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:08.906605 28040 ts_tablet_manager.cc:1397] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:08.906716 28040 raft_consensus.cc:357] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.906755 28040 raft_consensus.cc:383] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.906769 28040 raft_consensus.cc:738] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.906807 28040 consensus_queue.cc:260] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.906926 28040 ts_tablet_manager.cc:1428] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.906972 28040 tablet_bootstrap.cc:492] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:08.906980 28051 tablet_bootstrap.cc:492] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:08.907301 28051 tablet_bootstrap.cc:654] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.907359 28040 tablet_bootstrap.cc:654] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.908023 28042 tablet_bootstrap.cc:492] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:08.908213 28045 tablet_bootstrap.cc:492] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:08.908246 28045 ts_tablet_manager.cc:1397] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20250112 21:06:08.908344 28045 raft_consensus.cc:357] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.908382 28045 raft_consensus.cc:383] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.908394 28045 raft_consensus.cc:738] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.908429 28045 consensus_queue.cc:260] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.908469 28042 tablet_bootstrap.cc:654] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.908504 28045 ts_tablet_manager.cc:1428] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.908949 27382 tablet_service.cc:1467] Processing CreateTablet for tablet dbdd9ee5599c41a39985f0338da8dc38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=28f613c78d5c4e198a2ac99a1e0f360c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:08.909054 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdd9ee5599c41a39985f0338da8dc38. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:08.909157 28051 tablet_bootstrap.cc:492] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:08.909204 28051 ts_tablet_manager.cc:1397] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:08.909334 28042 tablet_bootstrap.cc:492] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:08.909334 28051 raft_consensus.cc:357] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.909385 28042 ts_tablet_manager.cc:1397] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:08.909392 28051 raft_consensus.cc:383] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.909413 28051 raft_consensus.cc:738] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.909452 28051 consensus_queue.cc:260] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.909536 28051 ts_tablet_manager.cc:1428] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.909531 28042 raft_consensus.cc:357] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.909593 28042 raft_consensus.cc:383] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.909623 28042 raft_consensus.cc:738] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.909677 28042 consensus_queue.cc:260] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.909796 28042 ts_tablet_manager.cc:1428] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:08.910108 28046 tablet_bootstrap.cc:492] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:08.910112 28045 tablet_bootstrap.cc:492] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:08.910156 28046 ts_tablet_manager.cc:1397] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250112 21:06:08.910281 28046 raft_consensus.cc:357] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.910321 28046 raft_consensus.cc:383] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.910333 28046 raft_consensus.cc:738] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.910368 28046 consensus_queue.cc:260] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:08.910452 28046 ts_tablet_manager.cc:1428] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.910506 28045 tablet_bootstrap.cc:654] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.910558 28041 tablet_bootstrap.cc:492] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:08.910622 28041 ts_tablet_manager.cc:1397] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250112 21:06:08.910666 28040 tablet_bootstrap.cc:492] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:08.910696 28040 ts_tablet_manager.cc:1397] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250112 21:06:08.910765 28041 raft_consensus.cc:357] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.910827 28041 raft_consensus.cc:383] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.910794 28040 raft_consensus.cc:357] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.910851 28041 raft_consensus.cc:738] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.910871 28040 raft_consensus.cc:383] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.910897 28040 raft_consensus.cc:738] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.910894 28041 consensus_queue.cc:260] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.910948 28040 consensus_queue.cc:260] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.910988 28041 ts_tablet_manager.cc:1428] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:08.911052 28041 tablet_bootstrap.cc:492] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:08.911049 28040 ts_tablet_manager.cc:1428] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.911123 28045 tablet_bootstrap.cc:492] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:08.911160 28045 ts_tablet_manager.cc:1397] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:08.911305 28045 raft_consensus.cc:357] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.911373 28045 raft_consensus.cc:383] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.911398 28045 raft_consensus.cc:738] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.911456 28045 consensus_queue.cc:260] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:08.911546 28045 ts_tablet_manager.cc:1428] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.911602 28041 tablet_bootstrap.cc:654] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:08.912164 28041 tablet_bootstrap.cc:492] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:08.912209 28041 ts_tablet_manager.cc:1397] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:08.912330 28041 raft_consensus.cc:357] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.912395 28041 raft_consensus.cc:383] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:08.912415 28041 raft_consensus.cc:738] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:08.912443 28041 consensus_queue.cc:260] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.912523 28041 ts_tablet_manager.cc:1428] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:08.939587 28095 raft_consensus.cc:491] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:08.939643 28095 raft_consensus.cc:513] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.939776 28095 leader_election.cc:290] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:08.939930 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f1198a8bb24c1aa3094bb61d35d39c" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:08.940008 27138 raft_consensus.cc:2463] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:08.939990 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f1198a8bb24c1aa3094bb61d35d39c" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:08.940083 27665 raft_consensus.cc:2463] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:08.940157 27337 leader_election.cc:304] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:08.940253 28095 raft_consensus.cc:2798] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:08.940292 28095 raft_consensus.cc:491] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:08.940322 28095 raft_consensus.cc:3054] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.940861 28095 raft_consensus.cc:513] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.940979 28095 leader_election.cc:290] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Requested vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:08.941109 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f1198a8bb24c1aa3094bb61d35d39c" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:08.941107 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f1198a8bb24c1aa3094bb61d35d39c" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:08.941174 27665 raft_consensus.cc:3054] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.941174 27138 raft_consensus.cc:3054] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:08.941809 27138 raft_consensus.cc:2463] T 38f1198a8bb24c1aa3094bb61d35d39c P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:08.941809 27665 raft_consensus.cc:2463] T 38f1198a8bb24c1aa3094bb61d35d39c P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:08.941983 27337 leader_election.cc:304] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:08.942082 28095 raft_consensus.cc:2798] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:08.942148 28095 raft_consensus.cc:695] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 LEADER]: Becoming Leader. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Running, Role: LEADER
I20250112 21:06:08.942194 28095 consensus_queue.cc:237] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:08.942685 26996 catalog_manager.cc:5526] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:09.172843 28095 raft_consensus.cc:491] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.172917 28095 raft_consensus.cc:513] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.173063 28095 leader_election.cc:290] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:09.173240 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbdd9ee5599c41a39985f0338da8dc38" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:09.173259 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbdd9ee5599c41a39985f0338da8dc38" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:09.173319 27138 raft_consensus.cc:2463] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:09.173364 27665 raft_consensus.cc:2463] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:09.173468 27337 leader_election.cc:304] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:09.173550 28095 raft_consensus.cc:2798] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.173584 28095 raft_consensus.cc:491] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.173616 28095 raft_consensus.cc:3054] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.174207 28095 raft_consensus.cc:513] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.174321 28095 leader_election.cc:290] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Requested vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:09.174438 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbdd9ee5599c41a39985f0338da8dc38" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:09.174450 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbdd9ee5599c41a39985f0338da8dc38" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:09.174512 27138 raft_consensus.cc:3054] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.174512 27665 raft_consensus.cc:3054] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.175216 27665 raft_consensus.cc:2463] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:09.175216 27138 raft_consensus.cc:2463] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:09.175371 27336 leader_election.cc:304] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:09.175453 28095 raft_consensus.cc:2798] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.175496 28095 raft_consensus.cc:695] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 LEADER]: Becoming Leader. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Running, Role: LEADER
I20250112 21:06:09.175531 28095 consensus_queue.cc:237] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.176067 26996 catalog_manager.cc:5526] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:09.271636 28167 raft_consensus.cc:491] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.271730 28167 raft_consensus.cc:513] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.271859 28167 leader_election.cc:290] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:09.272035 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68ba57a7dac345c29943b93e65c035bb" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:09.272033 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68ba57a7dac345c29943b93e65c035bb" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:09.272117 27798 raft_consensus.cc:2463] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:09.272119 27270 raft_consensus.cc:2463] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:09.272253 27468 leader_election.cc:304] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:09.272349 28167 raft_consensus.cc:2798] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.272399 28167 raft_consensus.cc:491] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.272414 28167 raft_consensus.cc:3054] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.273049 28167 raft_consensus.cc:513] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.273164 28167 leader_election.cc:290] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Requested vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:09.273316 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68ba57a7dac345c29943b93e65c035bb" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:09.273295 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68ba57a7dac345c29943b93e65c035bb" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:09.273377 27270 raft_consensus.cc:3054] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.273377 27798 raft_consensus.cc:3054] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.274021 27270 raft_consensus.cc:2463] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:09.274085 27798 raft_consensus.cc:2463] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:09.274197 27467 leader_election.cc:304] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:09.274295 28167 raft_consensus.cc:2798] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.274405 28167 raft_consensus.cc:695] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 LEADER]: Becoming Leader. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Running, Role: LEADER
I20250112 21:06:09.274452 28167 consensus_queue.cc:237] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.275043 26996 catalog_manager.cc:5526] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 reported cstate change: term changed from 0 to 1, leader changed from <none> to efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68). New cstate: current_term: 1 leader_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:09.363911 28095 consensus_queue.cc:1035] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.365123 28095 consensus_queue.cc:1035] T 38f1198a8bb24c1aa3094bb61d35d39c P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 21:06:09.422040 28289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:09.422180 28289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:09.423415 28289 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20250112 21:06:09.453562 27666 raft_consensus.cc:1270] T dbdd9ee5599c41a39985f0338da8dc38 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.453908 27137 raft_consensus.cc:1270] T dbdd9ee5599c41a39985f0338da8dc38 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.454131 27798 raft_consensus.cc:1270] T 68ba57a7dac345c29943b93e65c035bb P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.454313 28167 consensus_queue.cc:1035] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.454741 27270 raft_consensus.cc:1270] T 68ba57a7dac345c29943b93e65c035bb P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.454836 28095 consensus_queue.cc:1035] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.454928 28233 consensus_queue.cc:1035] T dbdd9ee5599c41a39985f0338da8dc38 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.455618 28287 consensus_queue.cc:1035] T 68ba57a7dac345c29943b93e65c035bb P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 21.6124 ms
time per row: 0.0108062 ms
I20250112 21:06:09.468137 26989 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:09.469563 27513 tablet_service.cc:1467] Processing CreateTablet for tablet 971a1261295445f8ba94f3d169000b0c (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.469607 27514 tablet_service.cc:1467] Processing CreateTablet for tablet 16af16332cde43c183d3f199a1e93b08 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.469686 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 971a1261295445f8ba94f3d169000b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.469728 27777 tablet_service.cc:1467] Processing CreateTablet for tablet 971a1261295445f8ba94f3d169000b0c (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.469863 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 971a1261295445f8ba94f3d169000b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.469873 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 16af16332cde43c183d3f199a1e93b08 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.469893 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 971a1261295445f8ba94f3d169000b0c (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.469970 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16af16332cde43c183d3f199a1e93b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.470016 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16af16332cde43c183d3f199a1e93b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.471006 27117 tablet_service.cc:1467] Processing CreateTablet for tablet 12b9602bb77948d5a0bb82bf2906cfa8 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.471107 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b9602bb77948d5a0bb82bf2906cfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.471132 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 971a1261295445f8ba94f3d169000b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.471174 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 12b9602bb77948d5a0bb82bf2906cfa8 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.471264 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b9602bb77948d5a0bb82bf2906cfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.471413 28350 tablet_bootstrap.cc:492] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:09.471714 28350 tablet_bootstrap.cc:654] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.472535 28352 tablet_bootstrap.cc:492] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:09.472535 27645 tablet_service.cc:1467] Processing CreateTablet for tablet 12b9602bb77948d5a0bb82bf2906cfa8 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.472632 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b9602bb77948d5a0bb82bf2906cfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.472817 28353 tablet_bootstrap.cc:492] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:09.473094 28352 tablet_bootstrap.cc:654] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.473198 28353 tablet_bootstrap.cc:654] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.473575 28350 tablet_bootstrap.cc:492] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:09.473605 28350 ts_tablet_manager.cc:1397] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:09.473675 28350 raft_consensus.cc:357] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.473724 28350 raft_consensus.cc:383] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.473745 28350 raft_consensus.cc:738] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.473749 28348 tablet_bootstrap.cc:492] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:09.473798 28350 consensus_queue.cc:260] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.473899 28350 ts_tablet_manager.cc:1428] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.473958 28350 tablet_bootstrap.cc:492] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:09.469738 27778 tablet_service.cc:1467] Processing CreateTablet for tablet 16af16332cde43c183d3f199a1e93b08 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f5edd98aa3d40cd906a5c381e75b741]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.474346 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16af16332cde43c183d3f199a1e93b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.474350 28350 tablet_bootstrap.cc:654] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.474364 28353 tablet_bootstrap.cc:492] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:09.474402 28353 ts_tablet_manager.cc:1397] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:09.474711 28353 raft_consensus.cc:357] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.474773 28353 raft_consensus.cc:383] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.474795 28353 raft_consensus.cc:738] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.474795 28352 tablet_bootstrap.cc:492] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:09.474841 28352 ts_tablet_manager.cc:1397] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:09.474854 28353 consensus_queue.cc:260] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.474967 28353 ts_tablet_manager.cc:1428] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:09.474988 28352 raft_consensus.cc:357] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.475054 28352 raft_consensus.cc:383] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.475080 28352 raft_consensus.cc:738] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.475150 28352 consensus_queue.cc:260] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.475244 28352 ts_tablet_manager.cc:1428] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.475291 28348 tablet_bootstrap.cc:654] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.475553 28350 tablet_bootstrap.cc:492] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:09.475600 28350 ts_tablet_manager.cc:1397] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:09.472396 28349 tablet_bootstrap.cc:492] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:09.475726 28350 raft_consensus.cc:357] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.475781 28350 raft_consensus.cc:383] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.475800 28350 raft_consensus.cc:738] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.475837 28350 consensus_queue.cc:260] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.475931 28350 ts_tablet_manager.cc:1428] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.476040 28349 tablet_bootstrap.cc:654] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.476377 28348 tablet_bootstrap.cc:492] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:09.476431 28348 ts_tablet_manager.cc:1397] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:09.476583 28356 tablet_bootstrap.cc:492] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:09.476593 28348 raft_consensus.cc:357] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.476657 28348 raft_consensus.cc:383] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.476697 28348 raft_consensus.cc:738] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.476752 28348 consensus_queue.cc:260] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.476869 28348 ts_tablet_manager.cc:1428] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.476919 28349 tablet_bootstrap.cc:492] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:09.476928 28348 tablet_bootstrap.cc:492] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:09.477020 28356 tablet_bootstrap.cc:654] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.476969 28349 ts_tablet_manager.cc:1397] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250112 21:06:09.477195 28349 raft_consensus.cc:357] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.477246 28349 raft_consensus.cc:383] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.477260 28349 raft_consensus.cc:738] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.477296 28349 consensus_queue.cc:260] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.477350 28348 tablet_bootstrap.cc:654] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.477404 28349 ts_tablet_manager.cc:1428] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.477452 28349 tablet_bootstrap.cc:492] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:09.477630 28356 tablet_bootstrap.cc:492] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:09.477675 28356 ts_tablet_manager.cc:1397] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.477824 28356 raft_consensus.cc:357] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.477890 28356 raft_consensus.cc:383] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.477910 28356 raft_consensus.cc:738] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.477955 28356 consensus_queue.cc:260] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.477978 28349 tablet_bootstrap.cc:654] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.478035 28356 ts_tablet_manager.cc:1428] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:09.478453 28348 tablet_bootstrap.cc:492] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:09.478510 28348 ts_tablet_manager.cc:1397] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:09.478647 28349 tablet_bootstrap.cc:492] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:09.478705 28349 ts_tablet_manager.cc:1397] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.478680 28348 raft_consensus.cc:357] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.478736 28348 raft_consensus.cc:383] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.478770 28348 raft_consensus.cc:738] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.478821 28348 consensus_queue.cc:260] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.478935 28348 ts_tablet_manager.cc:1428] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.478981 28349 raft_consensus.cc:357] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.479048 28349 raft_consensus.cc:383] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.479077 28349 raft_consensus.cc:738] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.479132 28349 consensus_queue.cc:260] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.479238 28349 ts_tablet_manager.cc:1428] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.531762 28287 raft_consensus.cc:491] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.531836 28287 raft_consensus.cc:513] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.532001 28287 leader_election.cc:290] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:09.532159 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "971a1261295445f8ba94f3d169000b0c" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:09.532183 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "971a1261295445f8ba94f3d169000b0c" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:09.532248 27270 raft_consensus.cc:2463] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:09.532261 27798 raft_consensus.cc:2463] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:09.532387 27467 leader_election.cc:304] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:09.532460 28287 raft_consensus.cc:2798] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.532488 28287 raft_consensus.cc:491] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.532512 28287 raft_consensus.cc:3054] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.533105 28287 raft_consensus.cc:513] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.533207 28287 leader_election.cc:290] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Requested vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:09.533336 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "971a1261295445f8ba94f3d169000b0c" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:09.533391 27270 raft_consensus.cc:3054] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.533390 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "971a1261295445f8ba94f3d169000b0c" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:09.533443 27798 raft_consensus.cc:3054] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.533892 27798 raft_consensus.cc:2463] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:09.533906 27270 raft_consensus.cc:2463] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:09.534021 27468 leader_election.cc:304] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:09.534116 28287 raft_consensus.cc:2798] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.534162 28287 raft_consensus.cc:695] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 LEADER]: Becoming Leader. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Running, Role: LEADER
I20250112 21:06:09.534214 28287 consensus_queue.cc:237] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:09.534667 26991 catalog_manager.cc:5526] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 reported cstate change: term changed from 0 to 1, leader changed from <none> to efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68). New cstate: current_term: 1 leader_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:09.592533 28166 raft_consensus.cc:491] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.592610 28166 raft_consensus.cc:513] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.592787 28166 leader_election.cc:290] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:09.592964 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b9602bb77948d5a0bb82bf2906cfa8" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:09.592972 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b9602bb77948d5a0bb82bf2906cfa8" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:09.593044 27401 raft_consensus.cc:2463] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:09.593050 27666 raft_consensus.cc:2463] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:09.593199 27071 leader_election.cc:304] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:09.593313 28166 raft_consensus.cc:2798] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.593362 28166 raft_consensus.cc:491] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.593395 28166 raft_consensus.cc:3054] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.593943 28166 raft_consensus.cc:513] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.594053 28166 leader_election.cc:290] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Requested vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:09.594234 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b9602bb77948d5a0bb82bf2906cfa8" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:09.594255 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b9602bb77948d5a0bb82bf2906cfa8" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:09.594321 27401 raft_consensus.cc:3054] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.594316 27666 raft_consensus.cc:3054] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.594831 27666 raft_consensus.cc:2463] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:09.594900 27401 raft_consensus.cc:2463] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:09.594985 27072 leader_election.cc:304] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:09.595077 28166 raft_consensus.cc:2798] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.595165 28166 raft_consensus.cc:695] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 LEADER]: Becoming Leader. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Running, Role: LEADER
I20250112 21:06:09.595213 28166 consensus_queue.cc:237] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:09.595688 26991 catalog_manager.cc:5526] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "90f636b130e64c04a7f6cdad4839ce33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:09.598774 28273 raft_consensus.cc:491] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.598832 28273 raft_consensus.cc:513] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.598974 28273 leader_election.cc:290] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:09.599126 27533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16af16332cde43c183d3f199a1e93b08" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:09.599157 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16af16332cde43c183d3f199a1e93b08" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:09.599216 27533 raft_consensus.cc:2463] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:09.599229 27798 raft_consensus.cc:2463] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:09.599359 27203 leader_election.cc:304] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:09.599442 28273 raft_consensus.cc:2798] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.599493 28273 raft_consensus.cc:491] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.599510 28273 raft_consensus.cc:3054] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.600023 28273 raft_consensus.cc:513] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.600127 28273 leader_election.cc:290] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:09.600252 27798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16af16332cde43c183d3f199a1e93b08" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:09.600323 27798 raft_consensus.cc:3054] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.600299 27533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16af16332cde43c183d3f199a1e93b08" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:09.600368 27533 raft_consensus.cc:3054] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.600841 27798 raft_consensus.cc:2463] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:09.600978 27533 raft_consensus.cc:2463] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:09.601001 27204 leader_election.cc:304] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:09.601076 28273 raft_consensus.cc:2798] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.601178 28273 raft_consensus.cc:695] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [term 1 LEADER]: Becoming Leader. State: Replica: c60c223c1269463181610419f08e11fc, State: Running, Role: LEADER
I20250112 21:06:09.601229 28273 consensus_queue.cc:237] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.601750 26991 catalog_manager.cc:5526] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c223c1269463181610419f08e11fc (127.26.82.66). New cstate: current_term: 1 leader_uuid: "c60c223c1269463181610419f08e11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: HEALTHY } } }
W20250112 21:06:09.686412 28368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:09.686596 28368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:09.687937 28368 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20250112 21:06:09.715149 27798 raft_consensus.cc:1270] T 971a1261295445f8ba94f3d169000b0c P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.715160 27270 raft_consensus.cc:1270] T 971a1261295445f8ba94f3d169000b0c P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.715420 28167 consensus_queue.cc:1035] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.715579 28287 consensus_queue.cc:1035] T 971a1261295445f8ba94f3d169000b0c P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.716224 27401 raft_consensus.cc:1270] T 12b9602bb77948d5a0bb82bf2906cfa8 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.716223 27666 raft_consensus.cc:1270] T 12b9602bb77948d5a0bb82bf2906cfa8 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.716625 28364 consensus_queue.cc:1035] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.716805 28364 consensus_queue.cc:1035] T 12b9602bb77948d5a0bb82bf2906cfa8 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.719765 28335 mvcc.cc:204] Tried to move back new op lower bound from 7113588611956068352 to 7113588611462189056. Current Snapshot: MvccSnapshot[applied={T|T < 7113588611956068352}]
I20250112 21:06:09.720886 27797 raft_consensus.cc:1270] T 16af16332cde43c183d3f199a1e93b08 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.721261 27533 raft_consensus.cc:1270] T 16af16332cde43c183d3f199a1e93b08 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:09.721653 28273 consensus_queue.cc:1035] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.723275 28366 consensus_queue.cc:1035] T 16af16332cde43c183d3f199a1e93b08 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:09.727526 28325 mvcc.cc:204] Tried to move back new op lower bound from 7113588611974311936 to 7113588611487039488. Current Snapshot: MvccSnapshot[applied={T|T < 7113588611974311936}]
INSERT report
rows total: 2000
time total: 24.2875 ms
time per row: 0.0121438 ms
I20250112 21:06:09.735239 26989 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:09.736616 27645 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa45653dab74f7e8f0b6122f6571714 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.736645 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa45653dab74f7e8f0b6122f6571714 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.736778 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa45653dab74f7e8f0b6122f6571714. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.736778 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa45653dab74f7e8f0b6122f6571714. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.737993 28353 tablet_bootstrap.cc:492] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:09.738034 27778 tablet_service.cc:1467] Processing CreateTablet for tablet 7b58561f353040e9a4298fadd2b099b0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.738155 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b58561f353040e9a4298fadd2b099b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.738366 28353 tablet_bootstrap.cc:654] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.738740 27117 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa45653dab74f7e8f0b6122f6571714 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.738770 27118 tablet_service.cc:1467] Processing CreateTablet for tablet d0c1af8ea3694f77bc65411415911df3 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.738821 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa45653dab74f7e8f0b6122f6571714. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.738875 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c1af8ea3694f77bc65411415911df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.739349 27646 tablet_service.cc:1467] Processing CreateTablet for tablet d0c1af8ea3694f77bc65411415911df3 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.739436 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c1af8ea3694f77bc65411415911df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.739670 28352 tablet_bootstrap.cc:492] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:09.739971 28352 tablet_bootstrap.cc:654] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.740252 28353 tablet_bootstrap.cc:492] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:09.740288 28353 ts_tablet_manager.cc:1397] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:09.740439 28353 raft_consensus.cc:357] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.740501 28353 raft_consensus.cc:383] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.740520 28353 raft_consensus.cc:738] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.740564 28353 consensus_queue.cc:260] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.740579 28352 tablet_bootstrap.cc:492] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:09.740589 28356 tablet_bootstrap.cc:492] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:09.740619 28352 ts_tablet_manager.cc:1397] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.740672 28353 ts_tablet_manager.cc:1428] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:09.740729 28349 tablet_bootstrap.cc:492] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:09.740775 28352 raft_consensus.cc:357] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.740828 28352 raft_consensus.cc:383] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.740867 28352 raft_consensus.cc:738] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.740926 28352 consensus_queue.cc:260] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.740948 28356 tablet_bootstrap.cc:654] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.741027 28352 ts_tablet_manager.cc:1428] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:09.741086 28352 tablet_bootstrap.cc:492] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:09.741151 28349 tablet_bootstrap.cc:654] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.741425 28356 tablet_bootstrap.cc:492] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:09.741467 28356 ts_tablet_manager.cc:1397] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.741478 28352 tablet_bootstrap.cc:654] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.741602 28356 raft_consensus.cc:357] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.741662 28356 raft_consensus.cc:383] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.741683 28356 raft_consensus.cc:738] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.741698 28349 tablet_bootstrap.cc:492] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:09.741736 28349 ts_tablet_manager.cc:1397] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.741720 28356 consensus_queue.cc:260] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.741811 28356 ts_tablet_manager.cc:1428] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:09.741865 28356 tablet_bootstrap.cc:492] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:09.741900 27382 tablet_service.cc:1467] Processing CreateTablet for tablet d0c1af8ea3694f77bc65411415911df3 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.741981 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c1af8ea3694f77bc65411415911df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.741961 28349 raft_consensus.cc:357] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.742029 28349 raft_consensus.cc:383] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.742051 28349 raft_consensus.cc:738] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.742100 28349 consensus_queue.cc:260] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.742177 28349 ts_tablet_manager.cc:1428] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:09.742236 28356 tablet_bootstrap.cc:654] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.742323 28352 tablet_bootstrap.cc:492] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:09.742359 28352 ts_tablet_manager.cc:1397] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.742506 28352 raft_consensus.cc:357] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.742561 28352 raft_consensus.cc:383] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.742584 28352 raft_consensus.cc:738] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.742565 27514 tablet_service.cc:1467] Processing CreateTablet for tablet 7b58561f353040e9a4298fadd2b099b0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.742614 28352 consensus_queue.cc:260] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.742687 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b58561f353040e9a4298fadd2b099b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.742725 28352 ts_tablet_manager.cc:1428] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.743042 28353 tablet_bootstrap.cc:492] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:09.743438 28353 tablet_bootstrap.cc:654] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.743933 28348 tablet_bootstrap.cc:492] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:09.743978 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 7b58561f353040e9a4298fadd2b099b0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef30d8f2fe4440a3bd8e1593d04ba23d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:09.744079 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b58561f353040e9a4298fadd2b099b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:09.744338 28348 tablet_bootstrap.cc:654] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.744365 28353 tablet_bootstrap.cc:492] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:09.744402 28353 ts_tablet_manager.cc:1397] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.744517 28353 raft_consensus.cc:357] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.744565 28353 raft_consensus.cc:383] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.744581 28353 raft_consensus.cc:738] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.744629 28353 consensus_queue.cc:260] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.744956 28353 ts_tablet_manager.cc:1428] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:09.744988 28348 tablet_bootstrap.cc:492] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:09.745034 28348 ts_tablet_manager.cc:1397] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.745170 28348 raft_consensus.cc:357] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.745236 28348 raft_consensus.cc:383] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.745260 28348 raft_consensus.cc:738] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.745302 28348 consensus_queue.cc:260] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.745401 28348 ts_tablet_manager.cc:1428] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.745460 28350 tablet_bootstrap.cc:492] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:09.745891 28350 tablet_bootstrap.cc:654] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:09.745913 28356 tablet_bootstrap.cc:492] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:09.745973 28356 ts_tablet_manager.cc:1397] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250112 21:06:09.746102 28356 raft_consensus.cc:357] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.746157 28356 raft_consensus.cc:383] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.746174 28356 raft_consensus.cc:738] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.746210 28356 consensus_queue.cc:260] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.746299 28356 ts_tablet_manager.cc:1428] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.746448 28350 tablet_bootstrap.cc:492] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:09.746498 28350 ts_tablet_manager.cc:1397] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:09.746711 28350 raft_consensus.cc:357] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.746781 28350 raft_consensus.cc:383] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:09.746798 28350 raft_consensus.cc:738] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:09.746858 28350 consensus_queue.cc:260] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.746973 28350 ts_tablet_manager.cc:1428] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:09.806581 28166 raft_consensus.cc:491] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.806667 28166 raft_consensus.cc:513] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.806823 28166 leader_election.cc:290] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:09.806967 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa45653dab74f7e8f0b6122f6571714" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:09.806988 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa45653dab74f7e8f0b6122f6571714" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:09.807037 27666 raft_consensus.cc:2463] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:09.807073 27401 raft_consensus.cc:2463] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:09.807230 27071 leader_election.cc:304] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:09.807334 28166 raft_consensus.cc:2798] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.807384 28166 raft_consensus.cc:491] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.807410 28166 raft_consensus.cc:3054] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.807994 28166 raft_consensus.cc:513] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.808125 28166 leader_election.cc:290] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Requested vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:09.808280 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa45653dab74f7e8f0b6122f6571714" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:09.808351 27401 raft_consensus.cc:3054] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.808322 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa45653dab74f7e8f0b6122f6571714" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:09.808379 27666 raft_consensus.cc:3054] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.809036 27401 raft_consensus.cc:2463] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:09.809036 27666 raft_consensus.cc:2463] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:09.809213 27071 leader_election.cc:304] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:09.809300 28166 raft_consensus.cc:2798] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.809363 28166 raft_consensus.cc:695] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 LEADER]: Becoming Leader. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Running, Role: LEADER
I20250112 21:06:09.809418 28166 consensus_queue.cc:237] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.809974 26992 catalog_manager.cc:5526] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "90f636b130e64c04a7f6cdad4839ce33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: HEALTHY } } }
I20250112 21:06:09.926671 28233 raft_consensus.cc:491] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.926749 28233 raft_consensus.cc:513] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.926920 28233 leader_election.cc:290] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:09.927073 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c1af8ea3694f77bc65411415911df3" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:09.927073 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c1af8ea3694f77bc65411415911df3" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:09.927156 27138 raft_consensus.cc:2463] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:09.927160 27666 raft_consensus.cc:2463] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 0.
I20250112 21:06:09.927306 27336 leader_election.cc:304] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:09.927400 28233 raft_consensus.cc:2798] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.927438 28233 raft_consensus.cc:491] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.927456 28233 raft_consensus.cc:3054] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.928040 28233 raft_consensus.cc:513] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.928157 28233 leader_election.cc:290] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Requested vote from peers 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:09.928299 27666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c1af8ea3694f77bc65411415911df3" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:09.928347 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c1af8ea3694f77bc65411415911df3" candidate_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:09.928390 27666 raft_consensus.cc:3054] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.928419 27138 raft_consensus.cc:3054] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.928970 27666 raft_consensus.cc:2463] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:09.929149 27336 leader_election.cc:304] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:09.929247 28233 raft_consensus.cc:2798] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.929304 28233 raft_consensus.cc:695] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 LEADER]: Becoming Leader. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Running, Role: LEADER
I20250112 21:06:09.929369 28233 consensus_queue.cc:237] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:09.929910 26992 catalog_manager.cc:5526] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:09.945151 27138 raft_consensus.cc:2463] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0de3679f8a4b8ebdc379e54e3a1d48 in term 1.
I20250112 21:06:09.953462 28104 raft_consensus.cc:491] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:09.953522 28104 raft_consensus.cc:513] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.953616 28104 leader_election.cc:290] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:09.953763 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b58561f353040e9a4298fadd2b099b0" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:09.953755 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b58561f353040e9a4298fadd2b099b0" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:09.953847 27270 raft_consensus.cc:2463] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:09.953858 27534 raft_consensus.cc:2463] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:09.953981 27731 leader_election.cc:304] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:09.954068 28104 raft_consensus.cc:2798] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:09.954116 28104 raft_consensus.cc:491] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:09.954140 28104 raft_consensus.cc:3054] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.954648 28104 raft_consensus.cc:513] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.954802 28104 leader_election.cc:290] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:09.954960 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b58561f353040e9a4298fadd2b099b0" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:09.954959 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b58561f353040e9a4298fadd2b099b0" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:09.955029 27534 raft_consensus.cc:3054] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.955029 27270 raft_consensus.cc:3054] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:09.955660 27534 raft_consensus.cc:2463] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:09.955660 27270 raft_consensus.cc:2463] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:09.955809 27731 leader_election.cc:304] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, c60c223c1269463181610419f08e11fc; no voters:
I20250112 21:06:09.955886 28104 raft_consensus.cc:2798] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:09.955988 28104 raft_consensus.cc:695] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Running, Role: LEADER
I20250112 21:06:09.956046 28104 consensus_queue.cc:237] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:09.956513 26992 catalog_manager.cc:5526] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4aa042eb06ac407fa827c735ea972020 (127.26.82.70). New cstate: current_term: 1 leader_uuid: "4aa042eb06ac407fa827c735ea972020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } }
W20250112 21:06:10.042029 28421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:10.042164 28421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:10.043449 28421 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20250112 21:06:10.070494 27534 raft_consensus.cc:1270] T 7b58561f353040e9a4298fadd2b099b0 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.070624 27270 raft_consensus.cc:1270] T 7b58561f353040e9a4298fadd2b099b0 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.070755 27666 raft_consensus.cc:1270] T d0c1af8ea3694f77bc65411415911df3 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.070811 28419 consensus_queue.cc:1035] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.071067 28275 consensus_queue.cc:1035] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.071863 27401 raft_consensus.cc:1270] T 0fa45653dab74f7e8f0b6122f6571714 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.072052 28364 consensus_queue.cc:1035] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.072129 28419 consensus_queue.cc:1035] T 7b58561f353040e9a4298fadd2b099b0 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.072299 27138 raft_consensus.cc:1270] T d0c1af8ea3694f77bc65411415911df3 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer dc0de3679f8a4b8ebdc379e54e3a1d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.072579 27666 raft_consensus.cc:1270] T 0fa45653dab74f7e8f0b6122f6571714 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.073019 28233 consensus_queue.cc:1035] T d0c1af8ea3694f77bc65411415911df3 P dc0de3679f8a4b8ebdc379e54e3a1d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.073798 28364 consensus_queue.cc:1035] T 0fa45653dab74f7e8f0b6122f6571714 P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.079468 28404 mvcc.cc:204] Tried to move back new op lower bound from 7113588613407236096 to 7113588612940316672. Current Snapshot: MvccSnapshot[applied={T|T < 7113588613407236096}]
INSERT report
rows total: 2000
time total: 30.8127 ms
time per row: 0.0154064 ms
I20250112 21:06:10.096879 26987 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:10.098630 27118 tablet_service.cc:1467] Processing CreateTablet for tablet a01e969f247f4262b452f3050b5794fb (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.098667 27513 tablet_service.cc:1467] Processing CreateTablet for tablet 9c092914ddef448d8ec3bfb9c590306b (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.098779 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a01e969f247f4262b452f3050b5794fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.098795 27778 tablet_service.cc:1467] Processing CreateTablet for tablet 7fea737a79c045249726534f561fa164 (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.098802 27514 tablet_service.cc:1467] Processing CreateTablet for tablet 7fea737a79c045249726534f561fa164 (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.098807 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c092914ddef448d8ec3bfb9c590306b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.098893 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fea737a79c045249726534f561fa164. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.098975 27250 tablet_service.cc:1467] Processing CreateTablet for tablet 7fea737a79c045249726534f561fa164 (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.099076 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fea737a79c045249726534f561fa164. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.099997 28352 tablet_bootstrap.cc:492] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:10.099997 27777 tablet_service.cc:1467] Processing CreateTablet for tablet 9c092914ddef448d8ec3bfb9c590306b (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.100083 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c092914ddef448d8ec3bfb9c590306b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.100203 28350 tablet_bootstrap.cc:492] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:10.100368 28352 tablet_bootstrap.cc:654] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.100497 28350 tablet_bootstrap.cc:654] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.100703 28348 tablet_bootstrap.cc:492] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:10.100806 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fea737a79c045249726534f561fa164. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.101033 28348 tablet_bootstrap.cc:654] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.101317 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 9c092914ddef448d8ec3bfb9c590306b (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.101405 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c092914ddef448d8ec3bfb9c590306b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.101550 28352 tablet_bootstrap.cc:492] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:10.101586 28352 ts_tablet_manager.cc:1397] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:10.101619 27646 tablet_service.cc:1467] Processing CreateTablet for tablet a01e969f247f4262b452f3050b5794fb (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.101702 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a01e969f247f4262b452f3050b5794fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.101701 28352 raft_consensus.cc:357] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.101749 28352 raft_consensus.cc:383] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.101784 28352 raft_consensus.cc:738] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.101841 28352 consensus_queue.cc:260] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.101939 28352 ts_tablet_manager.cc:1428] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.102248 28348 tablet_bootstrap.cc:492] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:10.102288 28348 ts_tablet_manager.cc:1397] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:10.102423 28348 raft_consensus.cc:357] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.102486 28348 raft_consensus.cc:383] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.102521 28348 raft_consensus.cc:738] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.102586 28348 consensus_queue.cc:260] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.102629 28349 tablet_bootstrap.cc:492] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:10.102650 28356 tablet_bootstrap.cc:492] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:10.102684 28348 ts_tablet_manager.cc:1428] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:10.102744 28348 tablet_bootstrap.cc:492] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:10.103003 28349 tablet_bootstrap.cc:654] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.103070 28356 tablet_bootstrap.cc:654] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.103135 28348 tablet_bootstrap.cc:654] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.103649 28356 tablet_bootstrap.cc:492] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:10.103693 28356 ts_tablet_manager.cc:1397] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.103838 28356 raft_consensus.cc:357] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.103909 28356 raft_consensus.cc:383] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.103929 28356 raft_consensus.cc:738] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.103966 28356 consensus_queue.cc:260] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.103996 27382 tablet_service.cc:1467] Processing CreateTablet for tablet a01e969f247f4262b452f3050b5794fb (DEFAULT_TABLE table=rebalance_test_table_9 [id=dc85ef877a234db2a057ee016f030b8a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.104107 28356 ts_tablet_manager.cc:1428] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.104130 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a01e969f247f4262b452f3050b5794fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.105424 28353 tablet_bootstrap.cc:492] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:10.105710 28353 tablet_bootstrap.cc:654] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.105834 28349 tablet_bootstrap.cc:492] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:10.105875 28349 ts_tablet_manager.cc:1397] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:10.106026 28349 raft_consensus.cc:357] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.106073 28349 raft_consensus.cc:383] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.106096 28349 raft_consensus.cc:738] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.106146 28349 consensus_queue.cc:260] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.106240 28353 tablet_bootstrap.cc:492] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:10.106247 28349 ts_tablet_manager.cc:1428] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.106289 28353 ts_tablet_manager.cc:1397] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.106318 28349 tablet_bootstrap.cc:492] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:10.106431 28353 raft_consensus.cc:357] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.106477 28353 raft_consensus.cc:383] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.106490 28353 raft_consensus.cc:738] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.106523 28353 consensus_queue.cc:260] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.106642 28353 ts_tablet_manager.cc:1428] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.106806 28349 tablet_bootstrap.cc:654] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.107517 28349 tablet_bootstrap.cc:492] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:10.107561 28349 ts_tablet_manager.cc:1397] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.107687 28349 raft_consensus.cc:357] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.107741 28349 raft_consensus.cc:383] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.107761 28349 raft_consensus.cc:738] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.107831 28349 consensus_queue.cc:260] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.107928 28349 ts_tablet_manager.cc:1428] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.120941 28350 tablet_bootstrap.cc:492] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:10.121003 28350 ts_tablet_manager.cc:1397] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250112 21:06:10.121173 28350 raft_consensus.cc:357] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.121227 28350 raft_consensus.cc:383] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.121248 28350 raft_consensus.cc:738] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.121291 28350 consensus_queue.cc:260] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.121603 28350 ts_tablet_manager.cc:1428] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:10.121673 28350 tablet_bootstrap.cc:492] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:10.122087 28350 tablet_bootstrap.cc:654] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.122524 28350 tablet_bootstrap.cc:492] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:10.122556 28350 ts_tablet_manager.cc:1397] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.122668 28350 raft_consensus.cc:357] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.122709 28350 raft_consensus.cc:383] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.122721 28350 raft_consensus.cc:738] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.122767 28350 consensus_queue.cc:260] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.122859 28350 ts_tablet_manager.cc:1428] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.128793 28348 tablet_bootstrap.cc:492] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:10.128847 28348 ts_tablet_manager.cc:1397] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.001s
I20250112 21:06:10.128973 28348 raft_consensus.cc:357] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.129014 28348 raft_consensus.cc:383] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.129025 28348 raft_consensus.cc:738] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.129055 28348 consensus_queue.cc:260] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.129137 28348 ts_tablet_manager.cc:1428] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.150060 28166 raft_consensus.cc:491] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.150116 28166 raft_consensus.cc:513] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.150254 28166 leader_election.cc:290] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:10.150372 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a01e969f247f4262b452f3050b5794fb" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:10.150388 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a01e969f247f4262b452f3050b5794fb" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59" is_pre_election: true
I20250112 21:06:10.150439 27401 raft_consensus.cc:2463] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:10.150449 27665 raft_consensus.cc:2463] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 0.
I20250112 21:06:10.150570 27071 leader_election.cc:304] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:10.150707 28166 raft_consensus.cc:2798] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.150751 28166 raft_consensus.cc:491] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.150779 28166 raft_consensus.cc:3054] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.151350 28166 raft_consensus.cc:513] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.151459 28166 leader_election.cc:290] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Requested vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69:38955)
I20250112 21:06:10.151573 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a01e969f247f4262b452f3050b5794fb" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:10.151623 27401 raft_consensus.cc:3054] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.151597 27665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a01e969f247f4262b452f3050b5794fb" candidate_uuid: "90f636b130e64c04a7f6cdad4839ce33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b470022ea814aa5bba1c6716c187a59"
I20250112 21:06:10.151654 27665 raft_consensus.cc:3054] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.152055 27401 raft_consensus.cc:2463] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:10.152109 27665 raft_consensus.cc:2463] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f636b130e64c04a7f6cdad4839ce33 in term 1.
I20250112 21:06:10.152307 27072 leader_election.cc:304] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:10.152397 28166 raft_consensus.cc:2798] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.152446 28166 raft_consensus.cc:695] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [term 1 LEADER]: Becoming Leader. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Running, Role: LEADER
I20250112 21:06:10.152493 28166 consensus_queue.cc:237] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } }
I20250112 21:06:10.152976 26992 catalog_manager.cc:5526] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "90f636b130e64c04a7f6cdad4839ce33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:10.200753 28104 raft_consensus.cc:491] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.200822 28104 raft_consensus.cc:513] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.200950 28104 leader_election.cc:290] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:10.201128 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fea737a79c045249726534f561fa164" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:10.201193 27534 raft_consensus.cc:2463] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:10.201153 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fea737a79c045249726534f561fa164" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:10.201238 27270 raft_consensus.cc:2463] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:10.201321 27731 leader_election.cc:304] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.201428 28104 raft_consensus.cc:2798] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.201471 28104 raft_consensus.cc:491] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.201496 28104 raft_consensus.cc:3054] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.202098 28104 raft_consensus.cc:513] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.202214 28104 leader_election.cc:290] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:10.202370 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fea737a79c045249726534f561fa164" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:10.202380 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fea737a79c045249726534f561fa164" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:10.202430 27534 raft_consensus.cc:3054] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.202433 27270 raft_consensus.cc:3054] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.202895 27534 raft_consensus.cc:2463] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:10.202901 27270 raft_consensus.cc:2463] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:10.203048 27731 leader_election.cc:304] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.203131 28104 raft_consensus.cc:2798] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.203181 28104 raft_consensus.cc:695] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Running, Role: LEADER
I20250112 21:06:10.203224 28104 consensus_queue.cc:237] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.203766 26992 catalog_manager.cc:5526] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4aa042eb06ac407fa827c735ea972020 (127.26.82.70). New cstate: current_term: 1 leader_uuid: "4aa042eb06ac407fa827c735ea972020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:10.254819 28287 raft_consensus.cc:491] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.254884 28287 raft_consensus.cc:513] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.255036 28287 leader_election.cc:290] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:10.255159 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c092914ddef448d8ec3bfb9c590306b" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:10.255187 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c092914ddef448d8ec3bfb9c590306b" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:10.255226 27270 raft_consensus.cc:2463] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:10.255250 27797 raft_consensus.cc:2463] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 0.
I20250112 21:06:10.255358 27467 leader_election.cc:304] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.255441 28287 raft_consensus.cc:2798] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.255499 28287 raft_consensus.cc:491] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.255519 28287 raft_consensus.cc:3054] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.258424 28287 raft_consensus.cc:513] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.258558 28287 leader_election.cc:290] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Requested vote from peers c60c223c1269463181610419f08e11fc (127.26.82.66:43929), 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441)
I20250112 21:06:10.258780 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c092914ddef448d8ec3bfb9c590306b" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:10.258849 27797 raft_consensus.cc:3054] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.258747 27270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c092914ddef448d8ec3bfb9c590306b" candidate_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:10.258985 27270 raft_consensus.cc:3054] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.259503 27797 raft_consensus.cc:2463] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:10.259636 27270 raft_consensus.cc:2463] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efed6ade5b3f4ee4aa1cb7885309a847 in term 1.
I20250112 21:06:10.259647 27468 leader_election.cc:304] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.259719 28287 raft_consensus.cc:2798] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.259804 28287 raft_consensus.cc:695] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 LEADER]: Becoming Leader. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Running, Role: LEADER
I20250112 21:06:10.259871 28287 consensus_queue.cc:237] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.260430 26992 catalog_manager.cc:5526] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 reported cstate change: term changed from 0 to 1, leader changed from <none> to efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68). New cstate: current_term: 1 leader_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: HEALTHY } } }
W20250112 21:06:10.358523 28473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:10.358664 28473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:10.359740 28473 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20250112 21:06:10.388983 27665 raft_consensus.cc:1270] T a01e969f247f4262b452f3050b5794fb P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.389029 27270 raft_consensus.cc:1270] T 7fea737a79c045249726534f561fa164 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.389155 27534 raft_consensus.cc:1270] T 7fea737a79c045249726534f561fa164 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.389156 27401 raft_consensus.cc:1270] T a01e969f247f4262b452f3050b5794fb P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 90f636b130e64c04a7f6cdad4839ce33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.389354 28364 consensus_queue.cc:1035] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.389354 28419 consensus_queue.cc:1035] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.389647 28419 consensus_queue.cc:1035] T 7fea737a79c045249726534f561fa164 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.390233 28364 consensus_queue.cc:1035] T a01e969f247f4262b452f3050b5794fb P 90f636b130e64c04a7f6cdad4839ce33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.390627 27797 raft_consensus.cc:1270] T 9c092914ddef448d8ec3bfb9c590306b P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.390848 28287 consensus_queue.cc:1035] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.391345 27269 raft_consensus.cc:1270] T 9c092914ddef448d8ec3bfb9c590306b P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer efed6ade5b3f4ee4aa1cb7885309a847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.391819 28471 consensus_queue.cc:1035] T 9c092914ddef448d8ec3bfb9c590306b P efed6ade5b3f4ee4aa1cb7885309a847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.395643 28324 mvcc.cc:204] Tried to move back new op lower bound from 7113588614710022144 to 7113588613952610304. Current Snapshot: MvccSnapshot[applied={T|T < 7113588614710022144}]
INSERT report
rows total: 2000
time total: 24.1232 ms
time per row: 0.0120616 ms
I20250112 21:06:10.407825 26987 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:10.409245 27646 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2f8866cca746c5b0b3b962e76b786e (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.409281 27118 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2f8866cca746c5b0b3b962e76b786e (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.409350 27117 tablet_service.cc:1467] Processing CreateTablet for tablet a9cc0053532348289e0db420ed5dca26 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.409394 27646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2f8866cca746c5b0b3b962e76b786e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.409394 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2f8866cca746c5b0b3b962e76b786e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.409466 27117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc0053532348289e0db420ed5dca26. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.410283 28352 tablet_bootstrap.cc:492] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:10.410569 28356 tablet_bootstrap.cc:492] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:10.410595 27645 tablet_service.cc:1467] Processing CreateTablet for tablet a9cc0053532348289e0db420ed5dca26 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.410678 28352 tablet_bootstrap.cc:654] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.410691 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc0053532348289e0db420ed5dca26. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.410993 28356 tablet_bootstrap.cc:654] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.411111 27514 tablet_service.cc:1467] Processing CreateTablet for tablet 42357d64f47c4dca86bc76886357ff7c (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.411159 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 42357d64f47c4dca86bc76886357ff7c (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.411211 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42357d64f47c4dca86bc76886357ff7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.411262 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42357d64f47c4dca86bc76886357ff7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.411399 27777 tablet_service.cc:1467] Processing CreateTablet for tablet 42357d64f47c4dca86bc76886357ff7c (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.411486 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42357d64f47c4dca86bc76886357ff7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.412263 28348 tablet_bootstrap.cc:492] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:10.412451 28350 tablet_bootstrap.cc:492] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:10.412743 27382 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2f8866cca746c5b0b3b962e76b786e (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.412859 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f2f8866cca746c5b0b3b962e76b786e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.412881 28348 tablet_bootstrap.cc:654] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.412881 28350 tablet_bootstrap.cc:654] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.413558 28348 tablet_bootstrap.cc:492] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:10.413601 28348 ts_tablet_manager.cc:1397] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:10.413722 28348 raft_consensus.cc:357] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.413762 28348 raft_consensus.cc:383] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.413777 28348 raft_consensus.cc:738] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.413777 28350 tablet_bootstrap.cc:492] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:10.413841 28350 ts_tablet_manager.cc:1397] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:10.413836 28348 consensus_queue.cc:260] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.413846 28353 tablet_bootstrap.cc:492] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:10.413890 27381 tablet_service.cc:1467] Processing CreateTablet for tablet a9cc0053532348289e0db420ed5dca26 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1b6d3fafef1c4fa197deac215ae12fba]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.413944 28348 ts_tablet_manager.cc:1428] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.413980 27381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc0053532348289e0db420ed5dca26. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.413973 28350 raft_consensus.cc:357] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.414028 28350 raft_consensus.cc:383] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.414052 28350 raft_consensus.cc:738] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.414108 28350 consensus_queue.cc:260] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.414191 28350 ts_tablet_manager.cc:1428] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.414280 28353 tablet_bootstrap.cc:654] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.414816 28353 tablet_bootstrap.cc:492] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:10.414847 28353 ts_tablet_manager.cc:1397] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.414919 28349 tablet_bootstrap.cc:492] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:10.414983 28353 raft_consensus.cc:357] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.415019 28353 raft_consensus.cc:383] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.415031 28353 raft_consensus.cc:738] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.415067 28353 consensus_queue.cc:260] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.415160 28353 ts_tablet_manager.cc:1428] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.415203 28353 tablet_bootstrap.cc:492] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:10.415205 28349 tablet_bootstrap.cc:654] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.415545 28353 tablet_bootstrap.cc:654] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.416172 28353 tablet_bootstrap.cc:492] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:10.416225 28353 ts_tablet_manager.cc:1397] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.416347 28353 raft_consensus.cc:357] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.416392 28353 raft_consensus.cc:383] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.416412 28353 raft_consensus.cc:738] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.416460 28353 consensus_queue.cc:260] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.416545 28353 ts_tablet_manager.cc:1428] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.416893 28349 tablet_bootstrap.cc:492] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:10.416949 28349 ts_tablet_manager.cc:1397] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:10.417105 28349 raft_consensus.cc:357] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.417183 28349 raft_consensus.cc:383] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.417204 28349 raft_consensus.cc:738] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.417250 28349 consensus_queue.cc:260] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.417373 28349 ts_tablet_manager.cc:1428] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.417611 28356 tablet_bootstrap.cc:492] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:10.417654 28356 ts_tablet_manager.cc:1397] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250112 21:06:10.417800 28356 raft_consensus.cc:357] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.417883 28356 raft_consensus.cc:383] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.417905 28356 raft_consensus.cc:738] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.417954 28356 consensus_queue.cc:260] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.418053 28356 ts_tablet_manager.cc:1428] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:10.418121 28356 tablet_bootstrap.cc:492] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:10.418627 28356 tablet_bootstrap.cc:654] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.419075 28356 tablet_bootstrap.cc:492] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:10.419122 28356 ts_tablet_manager.cc:1397] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.419137 28352 tablet_bootstrap.cc:492] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:10.419190 28352 ts_tablet_manager.cc:1397] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250112 21:06:10.419271 28356 raft_consensus.cc:357] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.419322 28356 raft_consensus.cc:383] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.419344 28356 raft_consensus.cc:738] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.419327 28352 raft_consensus.cc:357] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.419381 28352 raft_consensus.cc:383] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.419409 28352 raft_consensus.cc:738] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.419392 28356 consensus_queue.cc:260] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.419461 28352 consensus_queue.cc:260] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.419493 28356 ts_tablet_manager.cc:1428] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.419574 28352 ts_tablet_manager.cc:1428] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.419634 28352 tablet_bootstrap.cc:492] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:10.420046 28352 tablet_bootstrap.cc:654] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.420581 28352 tablet_bootstrap.cc:492] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:10.420624 28352 ts_tablet_manager.cc:1397] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.420773 28352 raft_consensus.cc:357] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.420842 28352 raft_consensus.cc:383] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.420863 28352 raft_consensus.cc:738] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.420912 28352 consensus_queue.cc:260] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.421013 28352 ts_tablet_manager.cc:1428] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.488833 28104 raft_consensus.cc:491] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.488903 28104 raft_consensus.cc:513] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.489028 28104 leader_election.cc:290] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:10.489225 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42357d64f47c4dca86bc76886357ff7c" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:10.489239 27269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42357d64f47c4dca86bc76886357ff7c" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:10.489317 27269 raft_consensus.cc:2463] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:10.489317 27534 raft_consensus.cc:2463] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:10.489444 27731 leader_election.cc:304] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.489531 28104 raft_consensus.cc:2798] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.489569 28104 raft_consensus.cc:491] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.489583 28104 raft_consensus.cc:3054] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.490118 28104 raft_consensus.cc:513] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.490212 28104 leader_election.cc:290] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:10.490348 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42357d64f47c4dca86bc76886357ff7c" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:10.490383 27269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42357d64f47c4dca86bc76886357ff7c" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:10.490428 27534 raft_consensus.cc:3054] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.490442 27269 raft_consensus.cc:3054] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.491108 27534 raft_consensus.cc:2463] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:10.491190 27269 raft_consensus.cc:2463] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:10.491271 27731 leader_election.cc:304] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.491359 28104 raft_consensus.cc:2798] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.491411 28104 raft_consensus.cc:695] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Running, Role: LEADER
I20250112 21:06:10.491464 28104 consensus_queue.cc:237] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } }
I20250112 21:06:10.492095 26988 catalog_manager.cc:5526] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4aa042eb06ac407fa827c735ea972020 (127.26.82.70). New cstate: current_term: 1 leader_uuid: "4aa042eb06ac407fa827c735ea972020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:10.502786 27876 raft_consensus.cc:491] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.502861 27876 raft_consensus.cc:513] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.502974 27876 leader_election.cc:290] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:10.503135 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc0053532348289e0db420ed5dca26" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:10.503163 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc0053532348289e0db420ed5dca26" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:10.503204 27138 raft_consensus.cc:2463] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:10.503237 27401 raft_consensus.cc:2463] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:10.503346 27601 leader_election.cc:304] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:10.503435 27876 raft_consensus.cc:2798] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.503487 27876 raft_consensus.cc:491] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.503520 27876 raft_consensus.cc:3054] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.504029 27876 raft_consensus.cc:513] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.504141 27876 leader_election.cc:290] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Requested vote from peers dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103), 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025)
I20250112 21:06:10.504308 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc0053532348289e0db420ed5dca26" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:10.504297 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc0053532348289e0db420ed5dca26" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:10.504371 27401 raft_consensus.cc:3054] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.504371 27138 raft_consensus.cc:3054] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.505060 27138 raft_consensus.cc:2463] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:10.505060 27401 raft_consensus.cc:2463] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:10.505246 27601 leader_election.cc:304] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:10.505336 27876 raft_consensus.cc:2798] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.505435 27876 raft_consensus.cc:695] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [term 1 LEADER]: Becoming Leader. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Running, Role: LEADER
I20250112 21:06:10.505489 27876 consensus_queue.cc:237] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } }
I20250112 21:06:10.505945 26988 catalog_manager.cc:5526] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69). New cstate: current_term: 1 leader_uuid: "9b470022ea814aa5bba1c6716c187a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:10.533767 27876 raft_consensus.cc:491] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.533833 27876 raft_consensus.cc:513] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.533967 27876 leader_election.cc:290] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:10.534125 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2f8866cca746c5b0b3b962e76b786e" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:10.534196 27401 raft_consensus.cc:2463] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:10.534162 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2f8866cca746c5b0b3b962e76b786e" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:10.534241 27138 raft_consensus.cc:2463] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:10.534354 27599 leader_election.cc:304] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:10.534452 27876 raft_consensus.cc:2798] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.534490 27876 raft_consensus.cc:491] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.534505 27876 raft_consensus.cc:3054] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.535015 27876 raft_consensus.cc:513] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.535125 27876 leader_election.cc:290] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Requested vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:10.535250 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2f8866cca746c5b0b3b962e76b786e" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:10.535269 27138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2f8866cca746c5b0b3b962e76b786e" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:10.535310 27401 raft_consensus.cc:3054] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.535323 27138 raft_consensus.cc:3054] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.535794 27401 raft_consensus.cc:2463] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:10.535795 27138 raft_consensus.cc:2463] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:10.535982 27599 leader_election.cc:304] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:10.536077 27876 raft_consensus.cc:2798] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.536144 27876 raft_consensus.cc:695] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [term 1 LEADER]: Becoming Leader. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Running, Role: LEADER
I20250112 21:06:10.536185 27876 consensus_queue.cc:237] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.536756 26988 catalog_manager.cc:5526] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69). New cstate: current_term: 1 leader_uuid: "9b470022ea814aa5bba1c6716c187a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } }
W20250112 21:06:10.628706 28526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:10.628849 28526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:10.630087 28526 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_10'
I20250112 21:06:10.658883 27269 raft_consensus.cc:1270] T 42357d64f47c4dca86bc76886357ff7c P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.658885 27534 raft_consensus.cc:1270] T 42357d64f47c4dca86bc76886357ff7c P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.659147 28419 consensus_queue.cc:1035] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.659291 28419 consensus_queue.cc:1035] T 42357d64f47c4dca86bc76886357ff7c P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.660534 27138 raft_consensus.cc:1270] T 5f2f8866cca746c5b0b3b962e76b786e P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.660765 28523 consensus_queue.cc:1035] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.661190 27401 raft_consensus.cc:1270] T 5f2f8866cca746c5b0b3b962e76b786e P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.661206 27137 raft_consensus.cc:1270] T a9cc0053532348289e0db420ed5dca26 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.661298 27402 raft_consensus.cc:1270] T a9cc0053532348289e0db420ed5dca26 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.662305 27876 consensus_queue.cc:1035] T 5f2f8866cca746c5b0b3b962e76b786e P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.662425 27876 consensus_queue.cc:1035] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.663169 27876 consensus_queue.cc:1035] T a9cc0053532348289e0db420ed5dca26 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.667606 28323 mvcc.cc:204] Tried to move back new op lower bound from 7113588615823708160 to 7113588615316451328. Current Snapshot: MvccSnapshot[applied={T|T < 7113588615823708160 or (T in {7113588615823708160})}]
INSERT report
rows total: 2000
time total: 25.1862 ms
time per row: 0.0125931 ms
I20250112 21:06:10.678133 26987 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35872:
name: "rebalance_test_table_11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:10.679368 27514 tablet_service.cc:1467] Processing CreateTablet for tablet 096cbf6ecfbf4a49b9bf23963c33e178 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.679414 27249 tablet_service.cc:1467] Processing CreateTablet for tablet 096cbf6ecfbf4a49b9bf23963c33e178 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.679500 27249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 096cbf6ecfbf4a49b9bf23963c33e178. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.679479 27513 tablet_service.cc:1467] Processing CreateTablet for tablet e26640330f3645b69d9cbfea4a854850 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.679533 27381 tablet_service.cc:1467] Processing CreateTablet for tablet 5bac83f6c752432e8f281fc247a0ab57 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.679491 27514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 096cbf6ecfbf4a49b9bf23963c33e178. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.679623 27381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bac83f6c752432e8f281fc247a0ab57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.679723 27513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26640330f3645b69d9cbfea4a854850. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.680776 27777 tablet_service.cc:1467] Processing CreateTablet for tablet 096cbf6ecfbf4a49b9bf23963c33e178 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.680804 28348 tablet_bootstrap.cc:492] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:10.680796 27250 tablet_service.cc:1467] Processing CreateTablet for tablet e26640330f3645b69d9cbfea4a854850 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.680881 27777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 096cbf6ecfbf4a49b9bf23963c33e178. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.680891 28350 tablet_bootstrap.cc:492] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:10.680891 27250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26640330f3645b69d9cbfea4a854850. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.681264 28348 tablet_bootstrap.cc:654] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.681455 28350 tablet_bootstrap.cc:654] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.681581 27118 tablet_service.cc:1467] Processing CreateTablet for tablet 5bac83f6c752432e8f281fc247a0ab57 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.681674 27118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bac83f6c752432e8f281fc247a0ab57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.682201 27778 tablet_service.cc:1467] Processing CreateTablet for tablet e26640330f3645b69d9cbfea4a854850 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.682292 27778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26640330f3645b69d9cbfea4a854850. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.682792 28352 tablet_bootstrap.cc:492] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33: Bootstrap starting.
I20250112 21:06:10.683151 28348 tablet_bootstrap.cc:492] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:10.683194 28352 tablet_bootstrap.cc:654] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.683207 28348 ts_tablet_manager.cc:1397] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:10.683418 28348 raft_consensus.cc:357] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.683475 28348 raft_consensus.cc:383] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.683494 28348 raft_consensus.cc:738] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.683506 28349 tablet_bootstrap.cc:492] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:10.683535 28348 consensus_queue.cc:260] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.683622 28348 ts_tablet_manager.cc:1428] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:10.683686 28348 tablet_bootstrap.cc:492] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847: Bootstrap starting.
I20250112 21:06:10.683872 28349 tablet_bootstrap.cc:654] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.683892 28352 tablet_bootstrap.cc:492] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33: No bootstrap required, opened a new log
I20250112 21:06:10.683938 28352 ts_tablet_manager.cc:1397] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:10.684032 28348 tablet_bootstrap.cc:654] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.684250 28352 raft_consensus.cc:357] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.684320 28352 raft_consensus.cc:383] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.684348 28352 raft_consensus.cc:738] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f636b130e64c04a7f6cdad4839ce33, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.684386 28352 consensus_queue.cc:260] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.684444 27645 tablet_service.cc:1467] Processing CreateTablet for tablet 5bac83f6c752432e8f281fc247a0ab57 (DEFAULT_TABLE table=rebalance_test_table_11 [id=9945e2faf6e74ff4b7b882ca065801ad]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:10.684479 28352 ts_tablet_manager.cc:1428] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:10.684540 27645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bac83f6c752432e8f281fc247a0ab57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:10.684741 28353 tablet_bootstrap.cc:492] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48: Bootstrap starting.
I20250112 21:06:10.685072 28353 tablet_bootstrap.cc:654] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.685374 28348 tablet_bootstrap.cc:492] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847: No bootstrap required, opened a new log
I20250112 21:06:10.685411 28348 ts_tablet_manager.cc:1397] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:10.685544 28348 raft_consensus.cc:357] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.685595 28348 raft_consensus.cc:383] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.685614 28348 raft_consensus.cc:738] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efed6ade5b3f4ee4aa1cb7885309a847, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.685627 28349 tablet_bootstrap.cc:492] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:10.685670 28349 ts_tablet_manager.cc:1397] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:10.685657 28348 consensus_queue.cc:260] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.685755 28348 ts_tablet_manager.cc:1428] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.685801 28349 raft_consensus.cc:357] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.685832 28356 tablet_bootstrap.cc:492] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59: Bootstrap starting.
I20250112 21:06:10.685858 28349 raft_consensus.cc:383] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.685871 28349 raft_consensus.cc:738] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.685909 28349 consensus_queue.cc:260] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.686012 28349 ts_tablet_manager.cc:1428] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.686165 28356 tablet_bootstrap.cc:654] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.686251 28353 tablet_bootstrap.cc:492] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48: No bootstrap required, opened a new log
I20250112 21:06:10.686292 28353 ts_tablet_manager.cc:1397] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:10.686440 28353 raft_consensus.cc:357] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.686486 28353 raft_consensus.cc:383] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.686506 28353 raft_consensus.cc:738] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0de3679f8a4b8ebdc379e54e3a1d48, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.686547 28353 consensus_queue.cc:260] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.686666 28353 ts_tablet_manager.cc:1428] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.686892 28349 tablet_bootstrap.cc:492] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020: Bootstrap starting.
I20250112 21:06:10.686944 28356 tablet_bootstrap.cc:492] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59: No bootstrap required, opened a new log
I20250112 21:06:10.686982 28356 ts_tablet_manager.cc:1397] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.687211 28356 raft_consensus.cc:357] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.687265 28356 raft_consensus.cc:383] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.687279 28356 raft_consensus.cc:738] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.687284 28349 tablet_bootstrap.cc:654] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.687310 28356 consensus_queue.cc:260] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.687404 28356 ts_tablet_manager.cc:1428] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:10.688040 28349 tablet_bootstrap.cc:492] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020: No bootstrap required, opened a new log
I20250112 21:06:10.688081 28349 ts_tablet_manager.cc:1397] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:10.688226 28349 raft_consensus.cc:357] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.688282 28349 raft_consensus.cc:383] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.688302 28349 raft_consensus.cc:738] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.688347 28349 consensus_queue.cc:260] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.688432 28349 ts_tablet_manager.cc:1428] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.688737 28350 tablet_bootstrap.cc:492] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:10.688781 28350 ts_tablet_manager.cc:1397] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20250112 21:06:10.688918 28350 raft_consensus.cc:357] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.688975 28350 raft_consensus.cc:383] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.688994 28350 raft_consensus.cc:738] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.689069 28350 consensus_queue.cc:260] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.689150 28350 ts_tablet_manager.cc:1428] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.689210 28350 tablet_bootstrap.cc:492] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc: Bootstrap starting.
I20250112 21:06:10.689674 28350 tablet_bootstrap.cc:654] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:10.690209 28350 tablet_bootstrap.cc:492] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc: No bootstrap required, opened a new log
I20250112 21:06:10.690263 28350 ts_tablet_manager.cc:1397] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:10.690394 28350 raft_consensus.cc:357] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.690451 28350 raft_consensus.cc:383] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:10.690470 28350 raft_consensus.cc:738] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c223c1269463181610419f08e11fc, State: Initialized, Role: FOLLOWER
I20250112 21:06:10.690521 28350 consensus_queue.cc:260] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.690603 28350 ts_tablet_manager.cc:1428] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:10.746194 28565 raft_consensus.cc:491] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.746255 28565 raft_consensus.cc:513] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.746345 28565 leader_election.cc:290] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:10.746497 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096cbf6ecfbf4a49b9bf23963c33e178" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:10.746522 27269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096cbf6ecfbf4a49b9bf23963c33e178" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc" is_pre_election: true
I20250112 21:06:10.746577 27269 raft_consensus.cc:2463] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:10.746573 27534 raft_consensus.cc:2463] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 0.
I20250112 21:06:10.746727 27731 leader_election.cc:304] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.746807 28565 raft_consensus.cc:2798] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.746857 28565 raft_consensus.cc:491] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.746886 28565 raft_consensus.cc:3054] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.747447 28565 raft_consensus.cc:513] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.747539 28565 leader_election.cc:290] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Requested vote from peers efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591), c60c223c1269463181610419f08e11fc (127.26.82.66:43929)
I20250112 21:06:10.747685 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096cbf6ecfbf4a49b9bf23963c33e178" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:10.747725 27269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096cbf6ecfbf4a49b9bf23963c33e178" candidate_uuid: "4aa042eb06ac407fa827c735ea972020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c223c1269463181610419f08e11fc"
I20250112 21:06:10.747748 27534 raft_consensus.cc:3054] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.747781 27269 raft_consensus.cc:3054] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.748281 27534 raft_consensus.cc:2463] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:10.748361 27269 raft_consensus.cc:2463] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4aa042eb06ac407fa827c735ea972020 in term 1.
I20250112 21:06:10.748473 27731 leader_election.cc:304] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aa042eb06ac407fa827c735ea972020, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.748553 28565 raft_consensus.cc:2798] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.748603 28565 raft_consensus.cc:695] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa042eb06ac407fa827c735ea972020, State: Running, Role: LEADER
I20250112 21:06:10.748649 28565 consensus_queue.cc:237] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } }
I20250112 21:06:10.749177 26995 catalog_manager.cc:5526] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4aa042eb06ac407fa827c735ea972020 (127.26.82.70). New cstate: current_term: 1 leader_uuid: "4aa042eb06ac407fa827c735ea972020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: HEALTHY } } }
I20250112 21:06:10.760697 28407 raft_consensus.cc:491] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.760754 28407 raft_consensus.cc:513] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.760866 28407 leader_election.cc:290] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:10.760977 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26640330f3645b69d9cbfea4a854850" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" is_pre_election: true
I20250112 21:06:10.761003 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26640330f3645b69d9cbfea4a854850" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020" is_pre_election: true
I20250112 21:06:10.761045 27534 raft_consensus.cc:2463] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:10.761062 27797 raft_consensus.cc:2463] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 0.
I20250112 21:06:10.761164 27203 leader_election.cc:304] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.761269 28407 raft_consensus.cc:2798] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.761309 28407 raft_consensus.cc:491] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.761356 28407 raft_consensus.cc:3054] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.761871 28407 raft_consensus.cc:513] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.762002 28407 leader_election.cc:290] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Requested vote from peers 4aa042eb06ac407fa827c735ea972020 (127.26.82.70:39441), efed6ade5b3f4ee4aa1cb7885309a847 (127.26.82.68:43591)
I20250112 21:06:10.762108 27797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26640330f3645b69d9cbfea4a854850" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aa042eb06ac407fa827c735ea972020"
I20250112 21:06:10.762122 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26640330f3645b69d9cbfea4a854850" candidate_uuid: "c60c223c1269463181610419f08e11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efed6ade5b3f4ee4aa1cb7885309a847"
I20250112 21:06:10.762167 27534 raft_consensus.cc:3054] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.762163 27797 raft_consensus.cc:3054] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.762632 27797 raft_consensus.cc:2463] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:10.762650 27534 raft_consensus.cc:2463] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c223c1269463181610419f08e11fc in term 1.
I20250112 21:06:10.762779 27203 leader_election.cc:304] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c60c223c1269463181610419f08e11fc, efed6ade5b3f4ee4aa1cb7885309a847; no voters:
I20250112 21:06:10.762876 28407 raft_consensus.cc:2798] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.762953 28407 raft_consensus.cc:695] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [term 1 LEADER]: Becoming Leader. State: Replica: c60c223c1269463181610419f08e11fc, State: Running, Role: LEADER
I20250112 21:06:10.762995 28407 consensus_queue.cc:237] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } }
I20250112 21:06:10.763510 26995 catalog_manager.cc:5526] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c223c1269463181610419f08e11fc (127.26.82.66). New cstate: current_term: 1 leader_uuid: "c60c223c1269463181610419f08e11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 } health_report { overall_health: UNKNOWN } } }
I20250112 21:06:10.801757 28557 raft_consensus.cc:491] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:10.801864 28557 raft_consensus.cc:513] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.802043 28557 leader_election.cc:290] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:10.802166 27137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bac83f6c752432e8f281fc247a0ab57" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33" is_pre_election: true
I20250112 21:06:10.802191 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bac83f6c752432e8f281fc247a0ab57" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" is_pre_election: true
I20250112 21:06:10.802246 27137 raft_consensus.cc:2463] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:10.802261 27401 raft_consensus.cc:2463] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 0.
I20250112 21:06:10.802390 27599 leader_election.cc:304] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b470022ea814aa5bba1c6716c187a59, dc0de3679f8a4b8ebdc379e54e3a1d48; no voters:
I20250112 21:06:10.802469 28557 raft_consensus.cc:2798] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:10.802505 28557 raft_consensus.cc:491] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:10.802520 28557 raft_consensus.cc:3054] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.803121 28557 raft_consensus.cc:513] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.803232 28557 leader_election.cc:290] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Requested vote from peers 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025), dc0de3679f8a4b8ebdc379e54e3a1d48 (127.26.82.67:44103)
I20250112 21:06:10.803346 27401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bac83f6c752432e8f281fc247a0ab57" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48"
I20250112 21:06:10.803341 27137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bac83f6c752432e8f281fc247a0ab57" candidate_uuid: "9b470022ea814aa5bba1c6716c187a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f636b130e64c04a7f6cdad4839ce33"
I20250112 21:06:10.803402 27137 raft_consensus.cc:3054] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.803403 27401 raft_consensus.cc:3054] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:10.803884 27137 raft_consensus.cc:2463] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:10.803925 27401 raft_consensus.cc:2463] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b470022ea814aa5bba1c6716c187a59 in term 1.
I20250112 21:06:10.804040 27601 leader_election.cc:304] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f636b130e64c04a7f6cdad4839ce33, 9b470022ea814aa5bba1c6716c187a59; no voters:
I20250112 21:06:10.804129 28557 raft_consensus.cc:2798] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:10.804194 28557 raft_consensus.cc:695] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [term 1 LEADER]: Becoming Leader. State: Replica: 9b470022ea814aa5bba1c6716c187a59, State: Running, Role: LEADER
I20250112 21:06:10.804243 28557 consensus_queue.cc:237] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } }
I20250112 21:06:10.804761 26995 catalog_manager.cc:5526] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b470022ea814aa5bba1c6716c187a59 (127.26.82.69). New cstate: current_term: 1 leader_uuid: "9b470022ea814aa5bba1c6716c187a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b470022ea814aa5bba1c6716c187a59" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 38955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 } health_report { overall_health: UNKNOWN } } }
W20250112 21:06:10.898129 28579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:10.898267 28579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:10.899384 28579 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_11'
I20250112 21:06:10.930507 27269 raft_consensus.cc:1270] T 096cbf6ecfbf4a49b9bf23963c33e178 P c60c223c1269463181610419f08e11fc [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.930567 27534 raft_consensus.cc:1270] T 096cbf6ecfbf4a49b9bf23963c33e178 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer 4aa042eb06ac407fa827c735ea972020: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.930838 28419 consensus_queue.cc:1035] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c223c1269463181610419f08e11fc" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.930963 28419 consensus_queue.cc:1035] T 096cbf6ecfbf4a49b9bf23963c33e178 P 4aa042eb06ac407fa827c735ea972020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.931269 27401 raft_consensus.cc:1270] T 5bac83f6c752432e8f281fc247a0ab57 P dc0de3679f8a4b8ebdc379e54e3a1d48 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.931427 27138 raft_consensus.cc:1270] T 5bac83f6c752432e8f281fc247a0ab57 P 90f636b130e64c04a7f6cdad4839ce33 [term 1 FOLLOWER]: Refusing update from remote peer 9b470022ea814aa5bba1c6716c187a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.931643 27533 raft_consensus.cc:1270] T e26640330f3645b69d9cbfea4a854850 P efed6ade5b3f4ee4aa1cb7885309a847 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.931636 27876 consensus_queue.cc:1035] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f636b130e64c04a7f6cdad4839ce33" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.931886 28576 consensus_queue.cc:1035] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "efed6ade5b3f4ee4aa1cb7885309a847" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.932063 27797 raft_consensus.cc:1270] T e26640330f3645b69d9cbfea4a854850 P 4aa042eb06ac407fa827c735ea972020 [term 1 FOLLOWER]: Refusing update from remote peer c60c223c1269463181610419f08e11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:10.933126 27876 consensus_queue.cc:1035] T 5bac83f6c752432e8f281fc247a0ab57 P 9b470022ea814aa5bba1c6716c187a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0de3679f8a4b8ebdc379e54e3a1d48" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.933547 28576 consensus_queue.cc:1035] T e26640330f3645b69d9cbfea4a854850 P c60c223c1269463181610419f08e11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aa042eb06ac407fa827c735ea972020" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:10.935590 28555 mvcc.cc:204] Tried to move back new op lower bound from 7113588616929177600 to 7113588616414449664. Current Snapshot: MvccSnapshot[applied={T|T < 7113588616929177600 or (T in {7113588616929177600})}]
INSERT report
rows total: 2000
time total: 30.3358 ms
time per row: 0.0151679 ms
I20250112 21:06:11.064191 27118 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:11.066747 27250 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.068949 27778 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.071043 27645 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.072086 27381 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:11.072777 27513 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.123759 27778 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.124804 27381 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:11.125871 27645 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.127229 27118 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:11.133189 27250 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.133213 27513 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.197654 27778 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.198576 27118 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:11.202657 27381 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:11.206161 27645 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.206682 27250 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.220790 27513 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.270627 27118 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:11.272291 27645 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.273816 27250 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.280164 27381 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:11.281443 27778 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.284740 27513 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.328106 27118 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:11.330747 27250 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.331326 27513 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.334650 27778 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.336328 27381 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:11.337719 27645 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.384501 27778 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.386790 27381 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:11.391891 27513 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.395288 27645 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.399252 27250 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:11.403179 27118 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
52ffc7d3d1d84bcea2dd92a719b8fe5c | 127.26.82.126:44733 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.26.82.84:43279 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
4aa042eb06ac407fa827c735ea972020 | 127.26.82.70:39441 | HEALTHY | /C | 6 | 0
90f636b130e64c04a7f6cdad4839ce33 | 127.26.82.65:40025 | HEALTHY | /A | 5 | 0
9b470022ea814aa5bba1c6716c187a59 | 127.26.82.69:38955 | HEALTHY | /C | 6 | 0
c60c223c1269463181610419f08e11fc | 127.26.82.66:43929 | HEALTHY | /A | 6 | 0
dc0de3679f8a4b8ebdc379e54e3a1d48 | 127.26.82.67:44103 | HEALTHY | /B | 7 | 0
efed6ade5b3f4ee4aa1cb7885309a847 | 127.26.82.68:43591 | HEALTHY | /B | 6 | 0
Tablet Server Location Summary
Location | Count
----------+---------
/B | 2
/C | 2
/A | 2
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 6 server(s) checked
local_ip_for_outbound_sockets | 127.26.82.65 | experimental | 127.26.82.65:40025
local_ip_for_outbound_sockets | 127.26.82.66 | experimental | 127.26.82.66:43929
local_ip_for_outbound_sockets | 127.26.82.67 | experimental | 127.26.82.67:44103
local_ip_for_outbound_sockets | 127.26.82.68 | experimental | 127.26.82.68:43591
local_ip_for_outbound_sockets | 127.26.82.69 | experimental | 127.26.82.69:38955
local_ip_for_outbound_sockets | 127.26.82.70 | experimental | 127.26.82.70:39441
never_fsync | true | unsafe,advanced | all 6 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 6 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 6 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 6 server(s) checked
server_dump_info_format | pb | hidden | all 6 server(s) checked
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.26.82.65:40025
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.26.82.66:43929
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.26.82.67:44103
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.26.82.68:43591
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.26.82.69:38955
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.LocationAwareRebalancingBasicTest.Basic.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.26.82.70:39441
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.26.82.84:43279 | all 6 server(s) checked
time_source | builtin | all 6 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 7 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------------------+----+---------+---------------+---------+------------+------------------+-------------
rebalance_test_table_0 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_10 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_11 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_5 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_6 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_7 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_8 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_9 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 18
First Quartile | 18
Median | 18
Third Quartile | 18
Maximum | 18
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 6
Tables | 12
Tablets | 36
Replicas | 108
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250112 21:06:11.411834 26953 log_verifier.cc:126] Checking tablet 096cbf6ecfbf4a49b9bf23963c33e178
I20250112 21:06:11.430462 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 096cbf6ecfbf4a49b9bf23963c33e178
I20250112 21:06:11.430519 26953 log_verifier.cc:126] Checking tablet 0fa45653dab74f7e8f0b6122f6571714
I20250112 21:06:11.448164 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 0fa45653dab74f7e8f0b6122f6571714
I20250112 21:06:11.448215 26953 log_verifier.cc:126] Checking tablet 1108d4d8d0154ebe86fa0c119480f389
I20250112 21:06:11.465775 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1108d4d8d0154ebe86fa0c119480f389
I20250112 21:06:11.465840 26953 log_verifier.cc:126] Checking tablet 12b9602bb77948d5a0bb82bf2906cfa8
I20250112 21:06:11.483573 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 12b9602bb77948d5a0bb82bf2906cfa8
I20250112 21:06:11.483628 26953 log_verifier.cc:126] Checking tablet 16af16332cde43c183d3f199a1e93b08
I20250112 21:06:11.502667 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 16af16332cde43c183d3f199a1e93b08
I20250112 21:06:11.502722 26953 log_verifier.cc:126] Checking tablet 1795c2e5a6f64569a0f8baf08b44ff1e
I20250112 21:06:11.519697 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1795c2e5a6f64569a0f8baf08b44ff1e
I20250112 21:06:11.519754 26953 log_verifier.cc:126] Checking tablet 2736e58da0e64362960fc8cc8612ecd2
I20250112 21:06:11.537135 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 2736e58da0e64362960fc8cc8612ecd2
I20250112 21:06:11.537192 26953 log_verifier.cc:126] Checking tablet 362bbb60e761455c906dfdb0ff0e6128
I20250112 21:06:11.554157 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 362bbb60e761455c906dfdb0ff0e6128
I20250112 21:06:11.554229 26953 log_verifier.cc:126] Checking tablet 377842729046491aa373c2c9244804c9
I20250112 21:06:11.571219 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 377842729046491aa373c2c9244804c9
I20250112 21:06:11.571280 26953 log_verifier.cc:126] Checking tablet 38f1198a8bb24c1aa3094bb61d35d39c
I20250112 21:06:11.588332 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 38f1198a8bb24c1aa3094bb61d35d39c
I20250112 21:06:11.588387 26953 log_verifier.cc:126] Checking tablet 3abd309fa6ac4e5d8efd743e96f2998d
I20250112 21:06:11.605567 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 3abd309fa6ac4e5d8efd743e96f2998d
I20250112 21:06:11.605628 26953 log_verifier.cc:126] Checking tablet 42357d64f47c4dca86bc76886357ff7c
I20250112 21:06:11.623685 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 42357d64f47c4dca86bc76886357ff7c
I20250112 21:06:11.623773 26953 log_verifier.cc:126] Checking tablet 525c56c9e1d442a89ef2f6428fc0949d
I20250112 21:06:11.640468 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 525c56c9e1d442a89ef2f6428fc0949d
I20250112 21:06:11.640520 26953 log_verifier.cc:126] Checking tablet 536162d5101c4d3685df680155707da7
I20250112 21:06:11.656771 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 536162d5101c4d3685df680155707da7
I20250112 21:06:11.656816 26953 log_verifier.cc:126] Checking tablet 5bac83f6c752432e8f281fc247a0ab57
I20250112 21:06:11.673192 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 5bac83f6c752432e8f281fc247a0ab57
I20250112 21:06:11.673241 26953 log_verifier.cc:126] Checking tablet 5f2f8866cca746c5b0b3b962e76b786e
I20250112 21:06:11.690017 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 5f2f8866cca746c5b0b3b962e76b786e
I20250112 21:06:11.690069 26953 log_verifier.cc:126] Checking tablet 66f307b250124c46980183887a4b460d
I20250112 21:06:11.706429 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 66f307b250124c46980183887a4b460d
I20250112 21:06:11.706491 26953 log_verifier.cc:126] Checking tablet 68ba57a7dac345c29943b93e65c035bb
I20250112 21:06:11.723122 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 68ba57a7dac345c29943b93e65c035bb
I20250112 21:06:11.723177 26953 log_verifier.cc:126] Checking tablet 73a992ed65bc42c6b7c81e1ce36ce39d
I20250112 21:06:11.739676 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 73a992ed65bc42c6b7c81e1ce36ce39d
I20250112 21:06:11.739728 26953 log_verifier.cc:126] Checking tablet 7b58561f353040e9a4298fadd2b099b0
I20250112 21:06:11.756294 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 7b58561f353040e9a4298fadd2b099b0
I20250112 21:06:11.756347 26953 log_verifier.cc:126] Checking tablet 7fea737a79c045249726534f561fa164
I20250112 21:06:11.773285 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 7fea737a79c045249726534f561fa164
I20250112 21:06:11.773340 26953 log_verifier.cc:126] Checking tablet 87d3010fd7704bda93e4fc592bc4b3f9
I20250112 21:06:11.790025 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 87d3010fd7704bda93e4fc592bc4b3f9
I20250112 21:06:11.790081 26953 log_verifier.cc:126] Checking tablet 971a1261295445f8ba94f3d169000b0c
I20250112 21:06:11.806566 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 971a1261295445f8ba94f3d169000b0c
I20250112 21:06:11.806622 26953 log_verifier.cc:126] Checking tablet 9c092914ddef448d8ec3bfb9c590306b
I20250112 21:06:11.823101 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 9c092914ddef448d8ec3bfb9c590306b
I20250112 21:06:11.823155 26953 log_verifier.cc:126] Checking tablet a01e969f247f4262b452f3050b5794fb
I20250112 21:06:11.840457 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet a01e969f247f4262b452f3050b5794fb
I20250112 21:06:11.840509 26953 log_verifier.cc:126] Checking tablet a9cc0053532348289e0db420ed5dca26
I20250112 21:06:11.857350 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet a9cc0053532348289e0db420ed5dca26
I20250112 21:06:11.857406 26953 log_verifier.cc:126] Checking tablet b2cfdb7868d6468f9bd3e0c6394278ba
I20250112 21:06:11.873737 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b2cfdb7868d6468f9bd3e0c6394278ba
I20250112 21:06:11.873792 26953 log_verifier.cc:126] Checking tablet b8c15dc68d6d453a8c8f353f91ac8b5b
I20250112 21:06:11.890862 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b8c15dc68d6d453a8c8f353f91ac8b5b
I20250112 21:06:11.890915 26953 log_verifier.cc:126] Checking tablet c2c3d077653440c88798984259616078
I20250112 21:06:11.907799 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c2c3d077653440c88798984259616078
I20250112 21:06:11.907864 26953 log_verifier.cc:126] Checking tablet c72146655e9f44688b0a6071aa841d5a
I20250112 21:06:11.924860 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c72146655e9f44688b0a6071aa841d5a
I20250112 21:06:11.924912 26953 log_verifier.cc:126] Checking tablet d0c1af8ea3694f77bc65411415911df3
I20250112 21:06:11.942763 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d0c1af8ea3694f77bc65411415911df3
I20250112 21:06:11.942826 26953 log_verifier.cc:126] Checking tablet dbdd9ee5599c41a39985f0338da8dc38
I20250112 21:06:11.960302 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet dbdd9ee5599c41a39985f0338da8dc38
I20250112 21:06:11.960351 26953 log_verifier.cc:126] Checking tablet de37ad95d2b74c5db78c175cf4ff8632
I20250112 21:06:11.978893 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet de37ad95d2b74c5db78c175cf4ff8632
I20250112 21:06:11.978948 26953 log_verifier.cc:126] Checking tablet dfd0bc2a10f64bc399ed609d672cae17
I20250112 21:06:11.996493 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet dfd0bc2a10f64bc399ed609d672cae17
I20250112 21:06:11.996559 26953 log_verifier.cc:126] Checking tablet e26640330f3645b69d9cbfea4a854850
I20250112 21:06:12.014456 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e26640330f3645b69d9cbfea4a854850
I20250112 21:06:12.014511 26953 log_verifier.cc:126] Checking tablet e60af3902e844f088d271b897635e987
I20250112 21:06:12.031818 26953 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e60af3902e844f088d271b897635e987
I20250112 21:06:12.036481 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 27056
I20250112 21:06:12.042524 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 27188
W20250112 21:06:12.045221 27733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.82.65:40025: connect: Connection refused (error 111)
W20250112 21:06:12.045631 27733 consensus_peers.cc:487] T c2c3d077653440c88798984259616078 P 4aa042eb06ac407fa827c735ea972020 -> Peer 90f636b130e64c04a7f6cdad4839ce33 (127.26.82.65:40025): Couldn't send request to peer 90f636b130e64c04a7f6cdad4839ce33. Status: Network error: Client connection negotiation failed: client connection to 127.26.82.65:40025: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:12.047974 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 27320
I20250112 21:06:12.053478 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 27452
I20250112 21:06:12.058512 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 27583
I20250112 21:06:12.063344 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 27715
I20250112 21:06:12.068585 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 26965
2025-01-12T21:06:12Z chronyd exiting
[ OK ] LocationAwareRebalancingBasicTest.Basic (10461 ms)
[----------] 1 test from LocationAwareRebalancingBasicTest (10461 ms total)
[----------] 1 test from RebalanceStartCriteriaTest
[ RUN ] RebalanceStartCriteriaTest.UnknownIgnoredTServer/0
I20250112 21:06:12.182916 26953 test_util.cc:274] Using random seed: -1298594945
I20250112 21:06:12.183398 26953 ts_itest-base.cc:112] Starting cluster with:
I20250112 21:06:12.183430 26953 ts_itest-base.cc:113] --------------
I20250112 21:06:12.183447 26953 ts_itest-base.cc:114] 5 tablet servers
I20250112 21:06:12.183462 26953 ts_itest-base.cc:115] 3 replicas per TS
I20250112 21:06:12.183478 26953 ts_itest-base.cc:116] --------------
2025-01-12T21:06:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-12T21:06:12Z Disabled control of system clock
I20250112 21:06:12.190656 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:33713
--webserver_interface=127.26.82.126
--webserver_port=0
--builtin_ntp_servers=127.26.82.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.82.126:33713
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20250112 21:06:12.264425 28895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:12.264572 28895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:12.264590 28895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:12.265800 28895 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:12.265843 28895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250112 21:06:12.265856 28895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:12.265868 28895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250112 21:06:12.265879 28895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250112 21:06:12.267233 28895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.82.126:33713
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:33713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.82.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.28895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:12.267390 28895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:12.267551 28895 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:12.272039 28903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.272208 28901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.272775 28900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:12.273919 28895 server_base.cc:1034] running on GCE node
I20250112 21:06:12.274129 28895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:12.274331 28895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:12.275461 28895 hybrid_clock.cc:648] HybridClock initialized: now 1736715972275448 us; error 26 us; skew 500 ppm
I20250112 21:06:12.276561 28895 webserver.cc:458] Webserver started at http://127.26.82.126:45727/ using document root <none> and password file <none>
I20250112 21:06:12.276769 28895 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:12.276819 28895 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:12.276947 28895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:12.277742 28895 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "5f3d35cdc1c647228df34c4266dfa4de"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.278030 28895 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "5f3d35cdc1c647228df34c4266dfa4de"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.279132 28895 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:12.279769 28909 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.279922 28895 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:12.279999 28895 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
uuid: "5f3d35cdc1c647228df34c4266dfa4de"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.280046 28895 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:12.295085 28895 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:12.295303 28895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:12.295399 28895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:12.298859 28895 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.126:33713
I20250112 21:06:12.298883 28961 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.126:33713 every 8 connection(s)
I20250112 21:06:12.299149 28895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250112 21:06:12.299628 28962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:12.301740 28962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de: Bootstrap starting.
I20250112 21:06:12.302256 28962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:12.302448 28962 log.cc:826] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:12.302934 28962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de: No bootstrap required, opened a new log
I20250112 21:06:12.304050 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 28895
I20250112 21:06:12.304121 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250112 21:06:12.304108 28962 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f3d35cdc1c647228df34c4266dfa4de" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 33713 } }
I20250112 21:06:12.304219 28962 raft_consensus.cc:383] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:12.304255 28962 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3d35cdc1c647228df34c4266dfa4de, State: Initialized, Role: FOLLOWER
I20250112 21:06:12.304445 28962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f3d35cdc1c647228df34c4266dfa4de" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 33713 } }
I20250112 21:06:12.304536 28962 raft_consensus.cc:397] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 21:06:12.304571 28962 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 21:06:12.304605 28962 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:12.305117 28962 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f3d35cdc1c647228df34c4266dfa4de" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 33713 } }
I20250112 21:06:12.305204 28962 leader_election.cc:304] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f3d35cdc1c647228df34c4266dfa4de; no voters:
I20250112 21:06:12.305349 28962 leader_election.cc:290] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 21:06:12.305400 28967 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:12.305495 28967 raft_consensus.cc:695] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [term 1 LEADER]: Becoming Leader. State: Replica: 5f3d35cdc1c647228df34c4266dfa4de, State: Running, Role: LEADER
I20250112 21:06:12.305562 28962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [sys.catalog]: configured and running, proceeding with master startup.
I20250112 21:06:12.305552 28967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f3d35cdc1c647228df34c4266dfa4de" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 33713 } }
I20250112 21:06:12.305845 28967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f3d35cdc1c647228df34c4266dfa4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f3d35cdc1c647228df34c4266dfa4de" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 33713 } } }
I20250112 21:06:12.305909 28967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:12.306036 28967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f3d35cdc1c647228df34c4266dfa4de. Latest consensus state: current_term: 1 leader_uuid: "5f3d35cdc1c647228df34c4266dfa4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f3d35cdc1c647228df34c4266dfa4de" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 33713 } } }
I20250112 21:06:12.306099 28967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:12.306288 28971 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 21:06:12.306732 28971 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 21:06:12.309159 28971 catalog_manager.cc:1348] Generated new cluster ID: d566b1b40fcf4943a3f6fd753bae3a1b
I20250112 21:06:12.309206 28971 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 21:06:12.322993 28971 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 21:06:12.323396 28971 catalog_manager.cc:1505] Loading token signing keys...
I20250112 21:06:12.330245 28971 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 5f3d35cdc1c647228df34c4266dfa4de: Generated new TSK 0
I20250112 21:06:12.330385 28971 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 21:06:12.332953 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.65:0
--local_ip_for_outbound_sockets=127.26.82.65
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--builtin_ntp_servers=127.26.82.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:12.407102 28986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:12.407255 28986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:12.407285 28986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:12.408486 28986 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:12.408525 28986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:12.408556 28986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.65
I20250112 21:06:12.409907 28986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--never_fsync=true
--heap_profile_path=/tmp/kudu.28986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.65
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:12.410115 28986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:12.410302 28986 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:12.414878 28992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.415617 28994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.416339 28991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:12.417176 28986 server_base.cc:1034] running on GCE node
I20250112 21:06:12.417335 28986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:12.417515 28986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:12.418640 28986 hybrid_clock.cc:648] HybridClock initialized: now 1736715972418624 us; error 34 us; skew 500 ppm
I20250112 21:06:12.425444 28986 webserver.cc:458] Webserver started at http://127.26.82.65:40585/ using document root <none> and password file <none>
I20250112 21:06:12.425643 28986 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:12.425693 28986 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:12.425783 28986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:12.426589 28986 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "feb58b5b575d4da8a3896f5e4d3b1934"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.426909 28986 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "feb58b5b575d4da8a3896f5e4d3b1934"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.428050 28986 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:12.428723 29000 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.428959 28986 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:12.429049 28986 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "feb58b5b575d4da8a3896f5e4d3b1934"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.429114 28986 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:12.436596 28986 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:12.436790 28986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:12.436924 28986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:12.437084 28986 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:12.437345 28986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:12.437374 28986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.437404 28986 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:12.437433 28986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.442714 28986 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.65:41605
I20250112 21:06:12.442766 29113 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.65:41605 every 8 connection(s)
I20250112 21:06:12.443009 28986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250112 21:06:12.447130 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 28986
I20250112 21:06:12.447207 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250112 21:06:12.447575 29114 heartbeater.cc:346] Connected to a master server at 127.26.82.126:33713
I20250112 21:06:12.447664 29114 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:12.447839 29114 heartbeater.cc:510] Master 127.26.82.126:33713 requested a full tablet report, sending...
I20250112 21:06:12.448199 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.66:0
--local_ip_for_outbound_sockets=127.26.82.66
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--builtin_ntp_servers=127.26.82.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:12.448252 28926 ts_manager.cc:194] Registered new tserver with Master: feb58b5b575d4da8a3896f5e4d3b1934 (127.26.82.65:41605)
I20250112 21:06:12.448825 28926 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.65:56395
W20250112 21:06:12.519227 29117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:12.519381 29117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:12.519400 29117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:12.520524 29117 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:12.520558 29117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:12.520579 29117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.66
I20250112 21:06:12.521950 29117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--never_fsync=true
--heap_profile_path=/tmp/kudu.29117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.66
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:12.522090 29117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:12.522258 29117 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:12.525969 29123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.526765 29122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.527420 29125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:12.528288 29117 server_base.cc:1034] running on GCE node
I20250112 21:06:12.528431 29117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:12.528609 29117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:12.529733 29117 hybrid_clock.cc:648] HybridClock initialized: now 1736715972529730 us; error 43 us; skew 500 ppm
I20250112 21:06:12.530781 29117 webserver.cc:458] Webserver started at http://127.26.82.66:36385/ using document root <none> and password file <none>
I20250112 21:06:12.530970 29117 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:12.531015 29117 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:12.531109 29117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:12.531968 29117 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "a3007c6d59e24c21b571cc4e78692823"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.532251 29117 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "a3007c6d59e24c21b571cc4e78692823"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.533406 29117 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250112 21:06:12.534003 29131 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.534152 29117 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:12.534219 29117 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "a3007c6d59e24c21b571cc4e78692823"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.534269 29117 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:12.548393 29117 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:12.548560 29117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:12.548673 29117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:12.548833 29117 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:12.549088 29117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:12.549117 29117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.549156 29117 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:12.549180 29117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.554306 29117 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.66:40503
I20250112 21:06:12.554364 29244 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.66:40503 every 8 connection(s)
I20250112 21:06:12.554596 29117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250112 21:06:12.559020 29245 heartbeater.cc:346] Connected to a master server at 127.26.82.126:33713
I20250112 21:06:12.559105 29245 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:12.559270 29245 heartbeater.cc:510] Master 127.26.82.126:33713 requested a full tablet report, sending...
I20250112 21:06:12.560211 28926 ts_manager.cc:194] Registered new tserver with Master: a3007c6d59e24c21b571cc4e78692823 (127.26.82.66:40503)
I20250112 21:06:12.560561 28926 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.66:59649
I20250112 21:06:12.561949 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 29117
I20250112 21:06:12.562034 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250112 21:06:12.562947 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.67:0
--local_ip_for_outbound_sockets=127.26.82.67
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--builtin_ntp_servers=127.26.82.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:12.636780 29248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:12.636931 29248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:12.636950 29248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:12.638190 29248 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:12.638231 29248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:12.638252 29248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.67
I20250112 21:06:12.639591 29248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--never_fsync=true
--heap_profile_path=/tmp/kudu.29248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.67
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:12.639744 29248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:12.639915 29248 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:12.644466 29256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.644695 29253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.645674 29254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:12.646524 29248 server_base.cc:1034] running on GCE node
I20250112 21:06:12.646659 29248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:12.646842 29248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:12.648020 29248 hybrid_clock.cc:648] HybridClock initialized: now 1736715972648006 us; error 31 us; skew 500 ppm
I20250112 21:06:12.649142 29248 webserver.cc:458] Webserver started at http://127.26.82.67:33683/ using document root <none> and password file <none>
I20250112 21:06:12.649312 29248 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:12.649359 29248 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:12.649452 29248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:12.650282 29248 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "2b4940d6133b48daa73e411bbbf80323"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.650570 29248 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "2b4940d6133b48daa73e411bbbf80323"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.651661 29248 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:12.652261 29262 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.652407 29248 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.652475 29248 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "2b4940d6133b48daa73e411bbbf80323"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.652529 29248 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:12.664840 29248 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:12.665015 29248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:12.665112 29248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:12.665268 29248 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:12.665526 29248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:12.665555 29248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.665584 29248 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:12.665609 29248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.670779 29248 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.67:39195
I20250112 21:06:12.670846 29375 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.67:39195 every 8 connection(s)
I20250112 21:06:12.671069 29248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250112 21:06:12.675315 29376 heartbeater.cc:346] Connected to a master server at 127.26.82.126:33713
I20250112 21:06:12.675390 29376 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:12.675554 29376 heartbeater.cc:510] Master 127.26.82.126:33713 requested a full tablet report, sending...
I20250112 21:06:12.675843 28926 ts_manager.cc:194] Registered new tserver with Master: 2b4940d6133b48daa73e411bbbf80323 (127.26.82.67:39195)
I20250112 21:06:12.676177 28926 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.67:42395
I20250112 21:06:12.676687 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 29248
I20250112 21:06:12.676784 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250112 21:06:12.677664 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.68:0
--local_ip_for_outbound_sockets=127.26.82.68
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--builtin_ntp_servers=127.26.82.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:12.750171 29379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:12.750324 29379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:12.750355 29379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:12.751581 29379 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:12.751622 29379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:12.751658 29379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.68
I20250112 21:06:12.753010 29379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--never_fsync=true
--heap_profile_path=/tmp/kudu.29379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.68
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:12.753216 29379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:12.753405 29379 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:12.757203 29385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.757936 29387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.758757 29384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:12.759627 29379 server_base.cc:1034] running on GCE node
I20250112 21:06:12.759771 29379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:12.759995 29379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:12.761133 29379 hybrid_clock.cc:648] HybridClock initialized: now 1736715972761116 us; error 26 us; skew 500 ppm
I20250112 21:06:12.762199 29379 webserver.cc:458] Webserver started at http://127.26.82.68:34611/ using document root <none> and password file <none>
I20250112 21:06:12.762377 29379 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:12.762434 29379 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:12.762523 29379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:12.763306 29379 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "1de54579304447478e9cfd4937eb7935"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.763599 29379 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "1de54579304447478e9cfd4937eb7935"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.764647 29379 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:12.765286 29393 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.765416 29379 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:12.765480 29379 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "1de54579304447478e9cfd4937eb7935"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.765530 29379 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:12.777190 29379 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:12.777367 29379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:12.777463 29379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:12.777612 29379 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:12.777868 29379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:12.777899 29379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.777930 29379 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:12.777959 29379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.783248 29379 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.68:37475
I20250112 21:06:12.783334 29506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.68:37475 every 8 connection(s)
I20250112 21:06:12.783577 29379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250112 21:06:12.787873 29507 heartbeater.cc:346] Connected to a master server at 127.26.82.126:33713
I20250112 21:06:12.787959 29507 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:12.788134 29507 heartbeater.cc:510] Master 127.26.82.126:33713 requested a full tablet report, sending...
I20250112 21:06:12.788498 28926 ts_manager.cc:194] Registered new tserver with Master: 1de54579304447478e9cfd4937eb7935 (127.26.82.68:37475)
I20250112 21:06:12.788898 28926 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.68:36161
I20250112 21:06:12.791198 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 29379
I20250112 21:06:12.791276 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250112 21:06:12.792176 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.69:0
--local_ip_for_outbound_sockets=127.26.82.69
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--builtin_ntp_servers=127.26.82.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:12.865543 29510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:12.865690 29510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:12.865710 29510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:12.866880 29510 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:12.866925 29510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:12.866947 29510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.69
I20250112 21:06:12.868278 29510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:33713
--never_fsync=true
--heap_profile_path=/tmp/kudu.29510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.69
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:12.868424 29510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:12.868587 29510 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:12.873057 29515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.873294 29518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:12.874162 29516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:12.874929 29510 server_base.cc:1034] running on GCE node
I20250112 21:06:12.875087 29510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:12.875244 29510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:12.876368 29510 hybrid_clock.cc:648] HybridClock initialized: now 1736715972876350 us; error 30 us; skew 500 ppm
I20250112 21:06:12.877449 29510 webserver.cc:458] Webserver started at http://127.26.82.69:41913/ using document root <none> and password file <none>
I20250112 21:06:12.877612 29510 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:12.877658 29510 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:12.877750 29510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:12.878523 29510 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "c8b618874785437fafc9f9e37dc7ba08"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.878790 29510 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "c8b618874785437fafc9f9e37dc7ba08"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.879832 29510 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:12.880369 29524 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.880498 29510 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:12.880558 29510 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "c8b618874785437fafc9f9e37dc7ba08"
format_stamp: "Formatted at 2025-01-12 21:06:12 on dist-test-slave-929r"
I20250112 21:06:12.880602 29510 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:12.892122 29510 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:12.892278 29510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:12.892374 29510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:12.892508 29510 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:12.892762 29510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:12.892792 29510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.892817 29510 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:12.892843 29510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:12.897894 29510 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.69:46299
I20250112 21:06:12.897929 29637 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.69:46299 every 8 connection(s)
I20250112 21:06:12.898182 29510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250112 21:06:12.902231 29638 heartbeater.cc:346] Connected to a master server at 127.26.82.126:33713
I20250112 21:06:12.902308 29638 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:12.902472 29638 heartbeater.cc:510] Master 127.26.82.126:33713 requested a full tablet report, sending...
I20250112 21:06:12.902742 28926 ts_manager.cc:194] Registered new tserver with Master: c8b618874785437fafc9f9e37dc7ba08 (127.26.82.69:46299)
I20250112 21:06:12.903119 28926 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.69:55577
I20250112 21:06:12.905916 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 29510
I20250112 21:06:12.905998 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceStartCriteriaTest.UnknownIgnoredTServer_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20250112 21:06:12.906872 26953 external_mini_cluster.cc:934] 5 TS(s) registered with all masters
I20250112 21:06:12.911579 28926 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51894:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250112 21:06:12.918498 29572 tablet_service.cc:1467] Processing CreateTablet for tablet 5d3c50deeddf43d8b7cfddaef6571eef (DEFAULT_TABLE table=TestTable [id=2d660b04b88c4734abdedd6157cc82c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:12.918599 29441 tablet_service.cc:1467] Processing CreateTablet for tablet 5d3c50deeddf43d8b7cfddaef6571eef (DEFAULT_TABLE table=TestTable [id=2d660b04b88c4734abdedd6157cc82c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:12.918768 29572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d3c50deeddf43d8b7cfddaef6571eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:12.918815 29441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d3c50deeddf43d8b7cfddaef6571eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:12.918756 29048 tablet_service.cc:1467] Processing CreateTablet for tablet 5d3c50deeddf43d8b7cfddaef6571eef (DEFAULT_TABLE table=TestTable [id=2d660b04b88c4734abdedd6157cc82c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:12.919014 29048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d3c50deeddf43d8b7cfddaef6571eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:12.921171 29655 tablet_bootstrap.cc:492] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934: Bootstrap starting.
I20250112 21:06:12.921438 29657 tablet_bootstrap.cc:492] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935: Bootstrap starting.
I20250112 21:06:12.921970 29657 tablet_bootstrap.cc:654] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:12.921970 29655 tablet_bootstrap.cc:654] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:12.922274 29655 log.cc:826] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:12.922277 29657 log.cc:826] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:12.922578 29656 tablet_bootstrap.cc:492] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08: Bootstrap starting.
I20250112 21:06:12.923023 29655 tablet_bootstrap.cc:492] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934: No bootstrap required, opened a new log
I20250112 21:06:12.923087 29655 ts_tablet_manager.cc:1397] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:12.923089 29656 tablet_bootstrap.cc:654] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:12.923314 29656 log.cc:826] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:12.923702 29657 tablet_bootstrap.cc:492] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935: No bootstrap required, opened a new log
I20250112 21:06:12.923750 29657 ts_tablet_manager.cc:1397] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:12.924639 29655 raft_consensus.cc:357] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:12.924772 29655 raft_consensus.cc:383] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:12.924811 29655 raft_consensus.cc:738] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb58b5b575d4da8a3896f5e4d3b1934, State: Initialized, Role: FOLLOWER
I20250112 21:06:12.924892 29655 consensus_queue.cc:260] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:12.925000 29656 tablet_bootstrap.cc:492] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08: No bootstrap required, opened a new log
I20250112 21:06:12.925066 29656 ts_tablet_manager.cc:1397] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:12.925146 29655 ts_tablet_manager.cc:1428] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250112 21:06:12.925240 29114 heartbeater.cc:502] Master 127.26.82.126:33713 was elected leader, sending a full tablet report...
I20250112 21:06:12.925698 29657 raft_consensus.cc:357] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:12.925904 29657 raft_consensus.cc:383] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:12.925937 29657 raft_consensus.cc:738] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de54579304447478e9cfd4937eb7935, State: Initialized, Role: FOLLOWER
I20250112 21:06:12.926069 29657 consensus_queue.cc:260] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:12.926321 29657 ts_tablet_manager.cc:1428] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20250112 21:06:12.926326 29507 heartbeater.cc:502] Master 127.26.82.126:33713 was elected leader, sending a full tablet report...
I20250112 21:06:12.927423 29656 raft_consensus.cc:357] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:12.927520 29656 raft_consensus.cc:383] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:12.927538 29656 raft_consensus.cc:738] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b618874785437fafc9f9e37dc7ba08, State: Initialized, Role: FOLLOWER
I20250112 21:06:12.927620 29656 consensus_queue.cc:260] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:12.927822 29656 ts_tablet_manager.cc:1428] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:12.927831 29638 heartbeater.cc:502] Master 127.26.82.126:33713 was elected leader, sending a full tablet report...
W20250112 21:06:12.944111 29115 tablet.cc:2367] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250112 21:06:13.034004 29508 tablet.cc:2367] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:13.057950 29663 raft_consensus.cc:491] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:13.058065 29663 raft_consensus.cc:513] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:13.058362 29663 leader_election.cc:290] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers feb58b5b575d4da8a3896f5e4d3b1934 (127.26.82.65:41605), 1de54579304447478e9cfd4937eb7935 (127.26.82.68:37475)
I20250112 21:06:13.061005 29068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3c50deeddf43d8b7cfddaef6571eef" candidate_uuid: "c8b618874785437fafc9f9e37dc7ba08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" is_pre_election: true
I20250112 21:06:13.061090 29461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3c50deeddf43d8b7cfddaef6571eef" candidate_uuid: "c8b618874785437fafc9f9e37dc7ba08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de54579304447478e9cfd4937eb7935" is_pre_election: true
I20250112 21:06:13.061136 29068 raft_consensus.cc:2463] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8b618874785437fafc9f9e37dc7ba08 in term 0.
I20250112 21:06:13.061220 29461 raft_consensus.cc:2463] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8b618874785437fafc9f9e37dc7ba08 in term 0.
I20250112 21:06:13.061316 29525 leader_election.cc:304] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8b618874785437fafc9f9e37dc7ba08, feb58b5b575d4da8a3896f5e4d3b1934; no voters:
I20250112 21:06:13.061468 29663 raft_consensus.cc:2798] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:13.061517 29663 raft_consensus.cc:491] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:13.061532 29663 raft_consensus.cc:3054] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:13.062139 29663 raft_consensus.cc:513] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:13.062255 29663 leader_election.cc:290] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [CANDIDATE]: Term 1 election: Requested vote from peers feb58b5b575d4da8a3896f5e4d3b1934 (127.26.82.65:41605), 1de54579304447478e9cfd4937eb7935 (127.26.82.68:37475)
I20250112 21:06:13.062399 29461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3c50deeddf43d8b7cfddaef6571eef" candidate_uuid: "c8b618874785437fafc9f9e37dc7ba08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de54579304447478e9cfd4937eb7935"
I20250112 21:06:13.062475 29461 raft_consensus.cc:3054] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:13.062461 29068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3c50deeddf43d8b7cfddaef6571eef" candidate_uuid: "c8b618874785437fafc9f9e37dc7ba08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feb58b5b575d4da8a3896f5e4d3b1934"
I20250112 21:06:13.062533 29068 raft_consensus.cc:3054] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:13.063129 29461 raft_consensus.cc:2463] T 5d3c50deeddf43d8b7cfddaef6571eef P 1de54579304447478e9cfd4937eb7935 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8b618874785437fafc9f9e37dc7ba08 in term 1.
I20250112 21:06:13.063313 29068 raft_consensus.cc:2463] T 5d3c50deeddf43d8b7cfddaef6571eef P feb58b5b575d4da8a3896f5e4d3b1934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8b618874785437fafc9f9e37dc7ba08 in term 1.
I20250112 21:06:13.063328 29526 leader_election.cc:304] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1de54579304447478e9cfd4937eb7935, c8b618874785437fafc9f9e37dc7ba08; no voters:
I20250112 21:06:13.063478 29663 raft_consensus.cc:2798] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:13.063617 29663 raft_consensus.cc:695] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [term 1 LEADER]: Becoming Leader. State: Replica: c8b618874785437fafc9f9e37dc7ba08, State: Running, Role: LEADER
I20250112 21:06:13.063700 29663 consensus_queue.cc:237] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } }
I20250112 21:06:13.064317 28925 catalog_manager.cc:5526] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8b618874785437fafc9f9e37dc7ba08 (127.26.82.69). New cstate: current_term: 1 leader_uuid: "c8b618874785437fafc9f9e37dc7ba08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 } } peers { permanent_uuid: "c8b618874785437fafc9f9e37dc7ba08" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 46299 } } peers { permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 } } }
I20250112 21:06:13.094499 26953 external_mini_cluster.cc:934] 5 TS(s) registered with all masters
I20250112 21:06:13.094951 26953 ts_itest-base.cc:233] Waiting for 1 tablets on tserver feb58b5b575d4da8a3896f5e4d3b1934 to finish bootstrapping
I20250112 21:06:13.098323 26953 ts_itest-base.cc:233] Waiting for 0 tablets on tserver a3007c6d59e24c21b571cc4e78692823 to finish bootstrapping
I20250112 21:06:13.101289 26953 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 2b4940d6133b48daa73e411bbbf80323 to finish bootstrapping
I20250112 21:06:13.104557 26953 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 1de54579304447478e9cfd4937eb7935 to finish bootstrapping
I20250112 21:06:13.107087 26953 ts_itest-base.cc:233] Waiting for 1 tablets on tserver c8b618874785437fafc9f9e37dc7ba08 to finish bootstrapping
W20250112 21:06:13.148640 29639 tablet.cc:2367] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:13.214275 29441 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.216408 29310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.216444 29572 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:13.216953 29179 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.217360 29048 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.258401 29441 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.258412 29179 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.259308 29572 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:13.259837 29310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.264137 29048 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.378659 29441 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.378777 29179 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.382618 29048 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.382710 29572 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:13.385042 29310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.420838 29310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.420989 29572 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:13.421123 29441 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.421263 29179 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.428663 29048 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.466590 29663 consensus_queue.cc:1035] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feb58b5b575d4da8a3896f5e4d3b1934" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 41605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:13.470733 29179 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.471455 29441 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.472620 29665 consensus_queue.cc:1035] T 5d3c50deeddf43d8b7cfddaef6571eef P c8b618874785437fafc9f9e37dc7ba08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1de54579304447478e9cfd4937eb7935" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 37475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:13.473268 29572 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:13.474100 29310 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.478500 29048 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:13.492233 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 28986
I20250112 21:06:13.497458 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 29117
I20250112 21:06:13.501627 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 29248
I20250112 21:06:13.505760 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 29379
I20250112 21:06:13.510375 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 29510
I20250112 21:06:13.515070 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 28895
2025-01-12T21:06:13Z chronyd exiting
[ OK ] RebalanceStartCriteriaTest.UnknownIgnoredTServer/0 (1352 ms)
[----------] 1 test from RebalanceStartCriteriaTest (1352 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/2
I20250112 21:06:13.535779 26953 test_util.cc:274] Using random seed: -1297242079
I20250112 21:06:13.536257 26953 ts_itest-base.cc:112] Starting cluster with:
I20250112 21:06:13.536288 26953 ts_itest-base.cc:113] --------------
I20250112 21:06:13.536302 26953 ts_itest-base.cc:114] 7 tablet servers
I20250112 21:06:13.536317 26953 ts_itest-base.cc:115] 2 replicas per TS
I20250112 21:06:13.536341 26953 ts_itest-base.cc:116] --------------
2025-01-12T21:06:13Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-12T21:06:13Z Disabled control of system clock
I20250112 21:06:13.543052 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:36745
--webserver_interface=127.26.82.126
--webserver_port=0
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.82.126:36745
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--tserver_unresponsive_timeout_ms=3000 with env {}
W20250112 21:06:13.614857 29848 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20250112 21:06:13.615003 29848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:13.615024 29848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:13.615047 29848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:13.616182 29848 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:13.616223 29848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250112 21:06:13.616243 29848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:13.616262 29848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250112 21:06:13.616286 29848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250112 21:06:13.617698 29848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.82.126:36745
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:36745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.82.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.29848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:13.617890 29848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:13.618105 29848 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:13.621680 29853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.622327 29856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.623215 29854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:13.623991 29848 server_base.cc:1034] running on GCE node
I20250112 21:06:13.624168 29848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:13.624356 29848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:13.625473 29848 hybrid_clock.cc:648] HybridClock initialized: now 1736715973625460 us; error 32 us; skew 500 ppm
I20250112 21:06:13.626600 29848 webserver.cc:458] Webserver started at http://127.26.82.126:45247/ using document root <none> and password file <none>
I20250112 21:06:13.626770 29848 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:13.626837 29848 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:13.626941 29848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:13.627732 29848 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "523c6484938d414ab64225a8ff19c01f"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.628021 29848 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "523c6484938d414ab64225a8ff19c01f"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.629109 29848 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:13.629694 29862 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.629853 29848 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:13.629920 29848 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
uuid: "523c6484938d414ab64225a8ff19c01f"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.629974 29848 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:13.638634 29848 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:13.638887 29848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:13.638996 29848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:13.642293 29848 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.126:36745
I20250112 21:06:13.642321 29914 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.126:36745 every 8 connection(s)
I20250112 21:06:13.642581 29848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250112 21:06:13.643101 29915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:13.645212 29915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f: Bootstrap starting.
I20250112 21:06:13.645680 29915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:13.645879 29915 log.cc:826] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:13.646175 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 29848
I20250112 21:06:13.646261 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250112 21:06:13.646452 29915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f: No bootstrap required, opened a new log
I20250112 21:06:13.648037 29915 raft_consensus.cc:357] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c6484938d414ab64225a8ff19c01f" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36745 } }
I20250112 21:06:13.648159 29915 raft_consensus.cc:383] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:13.648180 29915 raft_consensus.cc:738] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523c6484938d414ab64225a8ff19c01f, State: Initialized, Role: FOLLOWER
I20250112 21:06:13.648288 29915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c6484938d414ab64225a8ff19c01f" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36745 } }
I20250112 21:06:13.648365 29915 raft_consensus.cc:397] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 21:06:13.648406 29915 raft_consensus.cc:491] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 21:06:13.648460 29915 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:13.648983 29915 raft_consensus.cc:513] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c6484938d414ab64225a8ff19c01f" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36745 } }
I20250112 21:06:13.649071 29915 leader_election.cc:304] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 523c6484938d414ab64225a8ff19c01f; no voters:
I20250112 21:06:13.649206 29915 leader_election.cc:290] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 21:06:13.649398 29920 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:13.649433 29915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [sys.catalog]: configured and running, proceeding with master startup.
I20250112 21:06:13.649628 29920 raft_consensus.cc:695] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [term 1 LEADER]: Becoming Leader. State: Replica: 523c6484938d414ab64225a8ff19c01f, State: Running, Role: LEADER
I20250112 21:06:13.649762 29920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c6484938d414ab64225a8ff19c01f" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36745 } }
I20250112 21:06:13.650076 29921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "523c6484938d414ab64225a8ff19c01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c6484938d414ab64225a8ff19c01f" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36745 } } }
I20250112 21:06:13.650164 29921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:13.650310 29922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 523c6484938d414ab64225a8ff19c01f. Latest consensus state: current_term: 1 leader_uuid: "523c6484938d414ab64225a8ff19c01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523c6484938d414ab64225a8ff19c01f" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36745 } } }
I20250112 21:06:13.650334 29929 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 21:06:13.650367 29922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:13.650769 29929 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 21:06:13.652154 29929 catalog_manager.cc:1348] Generated new cluster ID: 671ba91f35de491aa9c50491115a19cf
I20250112 21:06:13.652200 29929 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 21:06:13.656271 29929 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 21:06:13.656939 29929 catalog_manager.cc:1505] Loading token signing keys...
I20250112 21:06:13.663992 29929 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 523c6484938d414ab64225a8ff19c01f: Generated new TSK 0
I20250112 21:06:13.664150 29929 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 21:06:13.672442 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.65:0
--local_ip_for_outbound_sockets=127.26.82.65
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:13.744933 29939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:13.745069 29939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:13.745086 29939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:13.746312 29939 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:13.746352 29939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:13.746374 29939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.65
I20250112 21:06:13.747722 29939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.29939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.65
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:13.747881 29939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:13.748093 29939 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:13.752774 29945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.752897 29944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.753738 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:13.754590 29939 server_base.cc:1034] running on GCE node
I20250112 21:06:13.754753 29939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:13.754992 29939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:13.756119 29939 hybrid_clock.cc:648] HybridClock initialized: now 1736715973756102 us; error 30 us; skew 500 ppm
I20250112 21:06:13.757205 29939 webserver.cc:458] Webserver started at http://127.26.82.65:43629/ using document root <none> and password file <none>
I20250112 21:06:13.757376 29939 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:13.757424 29939 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:13.757519 29939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:13.758322 29939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "cbe373da5cea408ea5c635c7f0164873"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.758590 29939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "cbe373da5cea408ea5c635c7f0164873"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.759688 29939 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:13.760283 29953 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.760434 29939 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.760495 29939 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "cbe373da5cea408ea5c635c7f0164873"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.760552 29939 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:13.771737 29939 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:13.771947 29939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:13.772063 29939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:13.772248 29939 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:13.772497 29939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:13.772527 29939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.772554 29939 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:13.772581 29939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.778014 29939 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.65:34885
I20250112 21:06:13.778064 30066 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.65:34885 every 8 connection(s)
I20250112 21:06:13.778324 29939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250112 21:06:13.783299 30067 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:13.783404 30067 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:13.783589 30067 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:13.784004 29879 ts_manager.cc:194] Registered new tserver with Master: cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:13.784646 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.65:52825
I20250112 21:06:13.786586 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 29939
I20250112 21:06:13.786660 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250112 21:06:13.787639 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.66:0
--local_ip_for_outbound_sockets=127.26.82.66
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:13.863400 30070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:13.863549 30070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:13.863567 30070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:13.864813 30070 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:13.864853 30070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:13.864876 30070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.66
I20250112 21:06:13.866170 30070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.30070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.66
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:13.866314 30070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:13.866479 30070 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:13.870705 30075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.870963 30078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.871695 30076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:13.872362 30070 server_base.cc:1034] running on GCE node
I20250112 21:06:13.872493 30070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:13.872649 30070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:13.873791 30070 hybrid_clock.cc:648] HybridClock initialized: now 1736715973873758 us; error 49 us; skew 500 ppm
I20250112 21:06:13.874846 30070 webserver.cc:458] Webserver started at http://127.26.82.66:36605/ using document root <none> and password file <none>
I20250112 21:06:13.875000 30070 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:13.875038 30070 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:13.875126 30070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:13.875945 30070 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "04cea27f437b471d8557d2fe649a7a7c"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.876242 30070 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "04cea27f437b471d8557d2fe649a7a7c"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.877403 30070 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250112 21:06:13.878027 30084 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.878208 30070 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:13.878268 30070 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "04cea27f437b471d8557d2fe649a7a7c"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.878324 30070 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:13.890676 30070 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:13.890910 30070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:13.891000 30070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:13.891177 30070 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:13.891424 30070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:13.891453 30070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.891481 30070 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:13.891505 30070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.896411 30070 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.66:42009
I20250112 21:06:13.896440 30197 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.66:42009 every 8 connection(s)
I20250112 21:06:13.896703 30070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250112 21:06:13.900640 30198 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:13.900758 30198 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:13.900936 30198 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:13.901355 29879 ts_manager.cc:194] Registered new tserver with Master: 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:13.901690 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.66:44343
I20250112 21:06:13.901732 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 30070
I20250112 21:06:13.901818 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250112 21:06:13.902706 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.67:0
--local_ip_for_outbound_sockets=127.26.82.67
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:13.977983 30201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:13.978125 30201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:13.978145 30201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:13.979354 30201 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:13.979393 30201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:13.979426 30201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.67
I20250112 21:06:13.980718 30201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.30201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.67
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:13.980904 30201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:13.981101 30201 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:13.984704 30206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.985369 30207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:13.986047 30209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:13.986721 30201 server_base.cc:1034] running on GCE node
I20250112 21:06:13.986881 30201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:13.987072 30201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:13.988195 30201 hybrid_clock.cc:648] HybridClock initialized: now 1736715973988180 us; error 30 us; skew 500 ppm
I20250112 21:06:13.989228 30201 webserver.cc:458] Webserver started at http://127.26.82.67:39287/ using document root <none> and password file <none>
I20250112 21:06:13.989392 30201 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:13.989436 30201 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:13.989531 30201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:13.990310 30201 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "b26f8d7d861f4663bd853aaec02b192c"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.990574 30201 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "b26f8d7d861f4663bd853aaec02b192c"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.991640 30201 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:13.992231 30215 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.992358 30201 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:13.992421 30201 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "b26f8d7d861f4663bd853aaec02b192c"
format_stamp: "Formatted at 2025-01-12 21:06:13 on dist-test-slave-929r"
I20250112 21:06:13.992472 30201 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:14.013005 30201 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:14.013233 30201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:14.013326 30201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:14.013486 30201 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:14.013741 30201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:14.013769 30201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.013795 30201 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:14.013820 30201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.018896 30201 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.67:39003
I20250112 21:06:14.018949 30328 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.67:39003 every 8 connection(s)
I20250112 21:06:14.019177 30201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250112 21:06:14.023306 30329 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:14.023378 30329 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:14.023509 30329 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:14.023797 29879 ts_manager.cc:194] Registered new tserver with Master: b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:14.024144 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.67:57465
I20250112 21:06:14.026660 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 30201
I20250112 21:06:14.026747 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250112 21:06:14.027679 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.68:0
--local_ip_for_outbound_sockets=127.26.82.68
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:14.099772 30332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:14.099926 30332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:14.099956 30332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:14.101150 30332 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:14.101192 30332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:14.101225 30332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.68
I20250112 21:06:14.102588 30332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.30332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.68
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:14.102779 30332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:14.102977 30332 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:14.106514 30338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.107403 30337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.107722 30340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:14.108495 30332 server_base.cc:1034] running on GCE node
I20250112 21:06:14.108640 30332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:14.108871 30332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:14.110013 30332 hybrid_clock.cc:648] HybridClock initialized: now 1736715974109995 us; error 30 us; skew 500 ppm
I20250112 21:06:14.111135 30332 webserver.cc:458] Webserver started at http://127.26.82.68:34025/ using document root <none> and password file <none>
I20250112 21:06:14.111300 30332 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:14.111344 30332 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:14.111430 30332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:14.112237 30332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "142e8c9809704200abdf893a5d130c23"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.112510 30332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "142e8c9809704200abdf893a5d130c23"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.113622 30332 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:14.114238 30346 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.114368 30332 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:14.114434 30332 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "142e8c9809704200abdf893a5d130c23"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.114487 30332 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:14.123327 30332 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:14.123548 30332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:14.123652 30332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:14.123818 30332 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:14.124058 30332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:14.124087 30332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.124115 30332 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:14.124142 30332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.130895 30332 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.68:42459
I20250112 21:06:14.130930 30459 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.68:42459 every 8 connection(s)
I20250112 21:06:14.131175 30332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250112 21:06:14.135185 30460 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:14.135259 30460 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:14.135387 30460 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:14.135666 29879 ts_manager.cc:194] Registered new tserver with Master: 142e8c9809704200abdf893a5d130c23 (127.26.82.68:42459)
I20250112 21:06:14.136044 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.68:58337
I20250112 21:06:14.141300 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 30332
I20250112 21:06:14.141396 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250112 21:06:14.142333 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.69:0
--local_ip_for_outbound_sockets=127.26.82.69
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:14.216404 30463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:14.216552 30463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:14.216569 30463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:14.217869 30463 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:14.217922 30463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:14.217947 30463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.69
I20250112 21:06:14.219357 30463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.30463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.69
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:14.219539 30463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:14.219764 30463 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:14.223884 30471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.224135 30468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.224917 30469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:14.225639 30463 server_base.cc:1034] running on GCE node
I20250112 21:06:14.225772 30463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:14.225953 30463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:14.227077 30463 hybrid_clock.cc:648] HybridClock initialized: now 1736715974227062 us; error 28 us; skew 500 ppm
I20250112 21:06:14.228173 30463 webserver.cc:458] Webserver started at http://127.26.82.69:44499/ using document root <none> and password file <none>
I20250112 21:06:14.228348 30463 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:14.228395 30463 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:14.228489 30463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:14.229331 30463 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "6e69a4b1e57242c2ba8e9e959c313e52"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.229605 30463 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "6e69a4b1e57242c2ba8e9e959c313e52"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.230654 30463 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:14.231222 30477 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.231361 30463 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:14.231439 30463 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "6e69a4b1e57242c2ba8e9e959c313e52"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.231490 30463 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:14.272157 30463 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:14.272399 30463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:14.272493 30463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:14.272687 30463 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:14.272964 30463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:14.272993 30463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.273022 30463 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:14.273048 30463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.278275 30463 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.69:41053
I20250112 21:06:14.278326 30590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.69:41053 every 8 connection(s)
I20250112 21:06:14.278553 30463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250112 21:06:14.282714 30591 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:14.282791 30591 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:14.282969 30591 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:14.283351 29879 ts_manager.cc:194] Registered new tserver with Master: 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69:41053)
I20250112 21:06:14.283681 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.69:41093
I20250112 21:06:14.286274 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 30463
I20250112 21:06:14.286343 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20250112 21:06:14.287194 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.70:0
--local_ip_for_outbound_sockets=127.26.82.70
--webserver_interface=127.26.82.70
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:14.359827 30594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:14.359972 30594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:14.359990 30594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:14.361236 30594 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:14.361285 30594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:14.361305 30594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.70
I20250112 21:06:14.362483 30594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.82.70
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.30594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.70
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:14.362620 30594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:14.362792 30594 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:14.366525 30599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.367197 30600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.368023 30602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:14.368862 30594 server_base.cc:1034] running on GCE node
I20250112 21:06:14.369009 30594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:14.369197 30594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:14.370328 30594 hybrid_clock.cc:648] HybridClock initialized: now 1736715974370316 us; error 33 us; skew 500 ppm
I20250112 21:06:14.371361 30594 webserver.cc:458] Webserver started at http://127.26.82.70:40969/ using document root <none> and password file <none>
I20250112 21:06:14.371533 30594 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:14.371573 30594 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:14.371655 30594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:14.372390 30594 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "b05fc98779d84f9589fe07913e1ba558"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.372645 30594 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "b05fc98779d84f9589fe07913e1ba558"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.373703 30594 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:14.374280 30608 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.374405 30594 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:14.374470 30594 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "b05fc98779d84f9589fe07913e1ba558"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.374523 30594 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:14.386359 30594 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:14.386584 30594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:14.386679 30594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:14.386862 30594 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:14.387130 30594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:14.387162 30594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.387192 30594 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:14.387220 30594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.392351 30594 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.70:36387
I20250112 21:06:14.392381 30721 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.70:36387 every 8 connection(s)
I20250112 21:06:14.392614 30594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20250112 21:06:14.396584 30722 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:14.396651 30722 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:14.396842 30722 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:14.397152 29879 ts_manager.cc:194] Registered new tserver with Master: b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387)
I20250112 21:06:14.397523 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.70:45229
I20250112 21:06:14.400846 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 30594
I20250112 21:06:14.400929 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20250112 21:06:14.401902 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.71:0
--local_ip_for_outbound_sockets=127.26.82.71
--webserver_interface=127.26.82.71
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:14.476205 30725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:14.476354 30725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:14.476372 30725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:14.477648 30725 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:14.477690 30725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:14.477712 30725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.71
I20250112 21:06:14.479116 30725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.82.71
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.30725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.71
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:14.479287 30725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:14.479456 30725 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:14.483107 30730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.483867 30731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:14.484706 30733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:14.485388 30725 server_base.cc:1034] running on GCE node
I20250112 21:06:14.485509 30725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:14.485682 30725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:14.486804 30725 hybrid_clock.cc:648] HybridClock initialized: now 1736715974486775 us; error 39 us; skew 500 ppm
I20250112 21:06:14.487948 30725 webserver.cc:458] Webserver started at http://127.26.82.71:44675/ using document root <none> and password file <none>
I20250112 21:06:14.488131 30725 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:14.488174 30725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:14.488273 30725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:14.489130 30725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "aa93c8a23781463188b9ecba6c3bbf79"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.489405 30725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "aa93c8a23781463188b9ecba6c3bbf79"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.490473 30725 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:14.491040 30739 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.491194 30725 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:14.491256 30725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "aa93c8a23781463188b9ecba6c3bbf79"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:14.491312 30725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:14.508605 30725 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:14.508845 30725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:14.508925 30725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:14.509074 30725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:14.509289 30725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:14.509313 30725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.509336 30725 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:14.509348 30725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:14.514348 30725 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.71:45067
I20250112 21:06:14.514384 30852 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.71:45067 every 8 connection(s)
I20250112 21:06:14.514623 30725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20250112 21:06:14.515646 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 30725
I20250112 21:06:14.515740 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20250112 21:06:14.519088 30853 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:14.519161 30853 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:14.519326 30853 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:14.519630 29879 ts_manager.cc:194] Registered new tserver with Master: aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:14.519955 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.71:49707
I20250112 21:06:14.527494 26953 external_mini_cluster.cc:934] 7 TS(s) registered with all masters
I20250112 21:06:14.532573 29879 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56216:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250112 21:06:14.539219 30787 tablet_service.cc:1467] Processing CreateTablet for tablet ba45544d32d54e369dd5bd0e379c0cb1 (DEFAULT_TABLE table=TestTable [id=9982a2b4bec0443585ea28fb19554c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:14.539515 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba45544d32d54e369dd5bd0e379c0cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:14.539458 30263 tablet_service.cc:1467] Processing CreateTablet for tablet ba45544d32d54e369dd5bd0e379c0cb1 (DEFAULT_TABLE table=TestTable [id=9982a2b4bec0443585ea28fb19554c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:14.539711 30263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba45544d32d54e369dd5bd0e379c0cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:14.542274 30868 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap starting.
I20250112 21:06:14.542284 30869 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:14.542941 30869 tablet_bootstrap.cc:654] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:14.542941 30868 tablet_bootstrap.cc:654] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:14.543241 30868 log.cc:826] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:14.543242 30869 log.cc:826] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:14.543861 30868 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: No bootstrap required, opened a new log
I20250112 21:06:14.543884 30869 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:14.543926 30868 ts_tablet_manager.cc:1397] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:14.543924 30869 ts_tablet_manager.cc:1397] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:14.546183 30869 raft_consensus.cc:357] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:14.546183 30868 raft_consensus.cc:357] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:14.546311 30868 raft_consensus.cc:383] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:14.546311 30869 raft_consensus.cc:383] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:14.546360 30868 raft_consensus.cc:738] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Initialized, Role: FOLLOWER
I20250112 21:06:14.546360 30869 raft_consensus.cc:738] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:14.546464 30869 consensus_queue.cc:260] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:14.546473 30868 consensus_queue.cc:260] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:14.546698 30868 ts_tablet_manager.cc:1428] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:14.546698 30869 ts_tablet_manager.cc:1428] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:14.546730 30329 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:14.546809 30853 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:14.559872 30873 raft_consensus.cc:491] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:14.559959 30873 raft_consensus.cc:513] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:14.560184 30873 leader_election.cc:290] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:14.562431 30807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba45544d32d54e369dd5bd0e379c0cb1" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79" is_pre_election: true
I20250112 21:06:14.562520 30807 raft_consensus.cc:2463] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 0.
I20250112 21:06:14.562706 30216 leader_election.cc:304] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: aa93c8a23781463188b9ecba6c3bbf79, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:14.562808 30873 raft_consensus.cc:2798] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:14.562884 30873 raft_consensus.cc:491] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:14.562906 30873 raft_consensus.cc:3054] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:14.563422 30873 raft_consensus.cc:513] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:14.563503 30873 leader_election.cc:290] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Requested vote from peers aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:14.563642 30807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba45544d32d54e369dd5bd0e379c0cb1" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79"
I20250112 21:06:14.563692 30807 raft_consensus.cc:3054] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:14.564268 30807 raft_consensus.cc:2463] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 1.
I20250112 21:06:14.564415 30216 leader_election.cc:304] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: aa93c8a23781463188b9ecba6c3bbf79, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:14.564499 30873 raft_consensus.cc:2798] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:14.564631 30873 raft_consensus.cc:695] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Becoming Leader. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:14.564728 30873 consensus_queue.cc:237] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:14.565259 29877 catalog_manager.cc:5526] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c reported cstate change: term changed from 0 to 1, leader changed from <none> to b26f8d7d861f4663bd853aaec02b192c (127.26.82.67). New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:14.572533 26953 external_mini_cluster.cc:934] 7 TS(s) registered with all masters
I20250112 21:06:14.573027 26953 ts_itest-base.cc:233] Waiting for 0 tablets on tserver cbe373da5cea408ea5c635c7f0164873 to finish bootstrapping
I20250112 21:06:14.576506 26953 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 04cea27f437b471d8557d2fe649a7a7c to finish bootstrapping
I20250112 21:06:14.579612 26953 ts_itest-base.cc:233] Waiting for 1 tablets on tserver b26f8d7d861f4663bd853aaec02b192c to finish bootstrapping
I20250112 21:06:14.582135 26953 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 142e8c9809704200abdf893a5d130c23 to finish bootstrapping
I20250112 21:06:14.585165 26953 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 6e69a4b1e57242c2ba8e9e959c313e52 to finish bootstrapping
I20250112 21:06:14.588210 26953 ts_itest-base.cc:233] Waiting for 0 tablets on tserver b05fc98779d84f9589fe07913e1ba558 to finish bootstrapping
I20250112 21:06:14.591130 26953 ts_itest-base.cc:233] Waiting for 1 tablets on tserver aa93c8a23781463188b9ecba6c3bbf79 to finish bootstrapping
I20250112 21:06:14.593669 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 30332
I20250112 21:06:14.598130 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 30463
I20250112 21:06:14.602159 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 30594
I20250112 21:06:14.606256 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 30725
W20250112 21:06:14.769874 30330 tablet.cc:2367] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:14.785457 30067 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:14.902454 30198 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
W20250112 21:06:15.086532 30216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.82.71:45067: connect: Connection refused (error 111)
W20250112 21:06:15.086971 30216 consensus_peers.cc:487] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Status: Network error: Client connection negotiation failed: client connection to 127.26.82.71:45067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 21:06:17.613806 30216 consensus_peers.cc:487] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Status: Network error: Client connection negotiation failed: client connection to 127.26.82.71:45067: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250112 21:06:18.360980 29877 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56216:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:18.366242 30132 tablet_service.cc:1467] Processing CreateTablet for tablet 879a5ea2638e4c8e8df85215be98a1b6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.366322 30130 tablet_service.cc:1467] Processing CreateTablet for tablet 58c99277feb2489aa208bb04c7136ba3 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.366245 30131 tablet_service.cc:1467] Processing CreateTablet for tablet f71bda1d7cfd4744825f7286fa592f2c (DEFAULT_TABLE table=rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.366611 30131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71bda1d7cfd4744825f7286fa592f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.366978 30132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879a5ea2638e4c8e8df85215be98a1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.366935 30001 tablet_service.cc:1467] Processing CreateTablet for tablet 879a5ea2638e4c8e8df85215be98a1b6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.366998 29999 tablet_service.cc:1467] Processing CreateTablet for tablet 58c99277feb2489aa208bb04c7136ba3 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.366937 30000 tablet_service.cc:1467] Processing CreateTablet for tablet f71bda1d7cfd4744825f7286fa592f2c (DEFAULT_TABLE table=rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.367216 30130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58c99277feb2489aa208bb04c7136ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.367252 30001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879a5ea2638e4c8e8df85215be98a1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.367336 30000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71bda1d7cfd4744825f7286fa592f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.367414 29999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58c99277feb2489aa208bb04c7136ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.369444 30906 tablet_bootstrap.cc:492] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:18.370052 30906 tablet_bootstrap.cc:654] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.370060 30907 tablet_bootstrap.cc:492] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:18.370602 30906 log.cc:826] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:18.370855 30907 tablet_bootstrap.cc:654] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.371130 30907 log.cc:826] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:18.371259 30906 tablet_bootstrap.cc:492] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:18.371325 30906 ts_tablet_manager.cc:1397] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:18.371865 30907 tablet_bootstrap.cc:492] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:18.371912 30907 ts_tablet_manager.cc:1397] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:18.372582 30906 raft_consensus.cc:357] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.372725 30906 raft_consensus.cc:383] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.372769 30906 raft_consensus.cc:738] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.372853 30906 consensus_queue.cc:260] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.373051 30906 ts_tablet_manager.cc:1428] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250112 21:06:18.373128 30906 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:18.373355 30907 raft_consensus.cc:357] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.373463 30907 raft_consensus.cc:383] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.373504 30906 tablet_bootstrap.cc:654] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.373526 30907 raft_consensus.cc:738] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.373631 30907 consensus_queue.cc:260] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.373847 30907 ts_tablet_manager.cc:1428] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250112 21:06:18.373942 30907 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:18.374074 30906 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:18.374121 30906 ts_tablet_manager.cc:1397] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:18.374293 30906 raft_consensus.cc:357] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.374356 30906 raft_consensus.cc:383] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.374375 30906 raft_consensus.cc:738] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.374370 30907 tablet_bootstrap.cc:654] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.374469 30906 consensus_queue.cc:260] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.374645 30906 ts_tablet_manager.cc:1428] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.374704 30906 tablet_bootstrap.cc:492] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:18.374861 30907 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:18.374905 30907 ts_tablet_manager.cc:1397] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:18.375033 30907 raft_consensus.cc:357] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.375113 30907 raft_consensus.cc:383] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.375108 30906 tablet_bootstrap.cc:654] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.375133 30907 raft_consensus.cc:738] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.375174 30907 consensus_queue.cc:260] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.375263 30907 ts_tablet_manager.cc:1428] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.375314 30907 tablet_bootstrap.cc:492] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:18.375666 30907 tablet_bootstrap.cc:654] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.375746 30906 tablet_bootstrap.cc:492] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:18.375784 30906 ts_tablet_manager.cc:1397] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:18.376016 30906 raft_consensus.cc:357] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.376139 30906 raft_consensus.cc:383] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.376173 30906 raft_consensus.cc:738] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.376217 30906 consensus_queue.cc:260] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.376304 30906 ts_tablet_manager.cc:1428] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.376322 30907 tablet_bootstrap.cc:492] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:18.376361 30907 ts_tablet_manager.cc:1397] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:18.376499 30907 raft_consensus.cc:357] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.376572 30907 raft_consensus.cc:383] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.376596 30907 raft_consensus.cc:738] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.376657 30907 consensus_queue.cc:260] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.376780 30907 ts_tablet_manager.cc:1428] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20250112 21:06:18.399565 30199 tablet.cc:2367] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:18.493899 30910 raft_consensus.cc:491] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:18.494002 30910 raft_consensus.cc:513] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.494314 30910 leader_election.cc:290] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:18.496966 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c99277feb2489aa208bb04c7136ba3" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873" is_pre_election: true
I20250112 21:06:18.497108 30021 raft_consensus.cc:2463] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 0.
I20250112 21:06:18.497323 30088 leader_election.cc:304] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.497470 30910 raft_consensus.cc:2798] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:18.497534 30910 raft_consensus.cc:491] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:18.497551 30910 raft_consensus.cc:3054] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.499080 30910 raft_consensus.cc:513] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.499192 30910 leader_election.cc:290] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Requested vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:18.499356 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c99277feb2489aa208bb04c7136ba3" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873"
I20250112 21:06:18.499423 30021 raft_consensus.cc:3054] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.503152 30021 raft_consensus.cc:2463] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 1.
I20250112 21:06:18.503304 30088 leader_election.cc:304] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.503388 30910 raft_consensus.cc:2798] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:18.503513 30910 raft_consensus.cc:695] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Leader. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:18.503600 30910 consensus_queue.cc:237] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.504281 29877 catalog_manager.cc:5526] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66). New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } }
W20250112 21:06:18.531127 30068 tablet.cc:2367] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:18.569666 30910 raft_consensus.cc:491] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:18.569747 30910 raft_consensus.cc:513] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.569882 30910 leader_election.cc:290] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:18.570115 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879a5ea2638e4c8e8df85215be98a1b6" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873" is_pre_election: true
I20250112 21:06:18.570238 30021 raft_consensus.cc:2463] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 0.
I20250112 21:06:18.570415 30088 leader_election.cc:304] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.570509 30910 raft_consensus.cc:2798] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:18.570556 30910 raft_consensus.cc:491] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:18.570580 30910 raft_consensus.cc:3054] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.572782 30910 raft_consensus.cc:513] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.572885 30910 leader_election.cc:290] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Requested vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:18.573107 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879a5ea2638e4c8e8df85215be98a1b6" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873"
I20250112 21:06:18.573182 30021 raft_consensus.cc:3054] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.584452 30021 raft_consensus.cc:2463] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 1.
I20250112 21:06:18.584669 30088 leader_election.cc:304] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.584774 30910 raft_consensus.cc:2798] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:18.584837 30910 raft_consensus.cc:695] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Leader. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:18.584883 30910 consensus_queue.cc:237] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.585440 29877 catalog_manager.cc:5526] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66). New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } }
I20250112 21:06:18.624729 30911 raft_consensus.cc:491] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:18.624804 30911 raft_consensus.cc:513] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.625089 30911 leader_election.cc:290] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:18.627645 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f71bda1d7cfd4744825f7286fa592f2c" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c" is_pre_election: true
I20250112 21:06:18.627741 30152 raft_consensus.cc:2463] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 0.
I20250112 21:06:18.627941 29956 leader_election.cc:304] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.628055 30911 raft_consensus.cc:2798] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:18.628116 30911 raft_consensus.cc:491] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:18.628141 30911 raft_consensus.cc:3054] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.631526 30911 raft_consensus.cc:513] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.631644 30911 leader_election.cc:290] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:18.631841 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f71bda1d7cfd4744825f7286fa592f2c" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:18.631923 30152 raft_consensus.cc:3054] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.642166 30152 raft_consensus.cc:2463] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 1.
I20250112 21:06:18.642360 29956 leader_election.cc:304] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.642447 30911 raft_consensus.cc:2798] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:18.642581 30911 raft_consensus.cc:695] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Becoming Leader. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Running, Role: LEADER
I20250112 21:06:18.642681 30911 consensus_queue.cc:237] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.643340 29877 catalog_manager.cc:5526] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbe373da5cea408ea5c635c7f0164873 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } }
W20250112 21:06:18.719425 30923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:18.719575 30923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:18.720768 30923 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20250112 21:06:18.753917 30021 raft_consensus.cc:1270] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:18.754045 30152 raft_consensus.cc:1270] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:18.753917 30020 raft_consensus.cc:1270] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:18.754279 30911 consensus_queue.cc:1035] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:18.754427 30917 consensus_queue.cc:1035] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:18.754686 30917 consensus_queue.cc:1035] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:18.757287 30957 mvcc.cc:204] Tried to move back new op lower bound from 7113588648973295616 to 7113588647951249408. Current Snapshot: MvccSnapshot[applied={T|T < 7113588648973295616}]
I20250112 21:06:18.757287 30955 mvcc.cc:204] Tried to move back new op lower bound from 7113588648973295616 to 7113588647951249408. Current Snapshot: MvccSnapshot[applied={T|T < 7113588648973295616}]
INSERT report
rows total: 2000
time total: 19.1789 ms
time per row: 0.00958944 ms
I20250112 21:06:18.767002 29876 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56216:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:18.768412 30132 tablet_service.cc:1467] Processing CreateTablet for tablet fd61ff11a25e482182da04059a77ed36 (DEFAULT_TABLE table=rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.768450 29999 tablet_service.cc:1467] Processing CreateTablet for tablet d0dde2209dbc4c9a9a874f32baff5c5d (DEFAULT_TABLE table=rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.768451 30263 tablet_service.cc:1467] Processing CreateTablet for tablet f2ad09a23ec4424187345c4914d19ac4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.768435 30001 tablet_service.cc:1467] Processing CreateTablet for tablet f2ad09a23ec4424187345c4914d19ac4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.768539 30132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd61ff11a25e482182da04059a77ed36. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.768569 30001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ad09a23ec4424187345c4914d19ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.768572 30263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ad09a23ec4424187345c4914d19ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.768649 29999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0dde2209dbc4c9a9a874f32baff5c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.769740 30907 tablet_bootstrap.cc:492] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:18.769840 30970 tablet_bootstrap.cc:492] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:18.769887 30262 tablet_service.cc:1467] Processing CreateTablet for tablet fd61ff11a25e482182da04059a77ed36 (DEFAULT_TABLE table=rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.769984 30262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd61ff11a25e482182da04059a77ed36. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.770197 30970 tablet_bootstrap.cc:654] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.770714 30970 tablet_bootstrap.cc:492] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:18.770756 30970 ts_tablet_manager.cc:1397] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:18.770843 30261 tablet_service.cc:1467] Processing CreateTablet for tablet d0dde2209dbc4c9a9a874f32baff5c5d (DEFAULT_TABLE table=rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:18.770923 30261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0dde2209dbc4c9a9a874f32baff5c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:18.770927 30970 raft_consensus.cc:357] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.771010 30970 raft_consensus.cc:383] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.771037 30970 raft_consensus.cc:738] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.771059 30907 tablet_bootstrap.cc:654] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.771073 30970 consensus_queue.cc:260] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.771183 30970 ts_tablet_manager.cc:1428] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.771268 30970 tablet_bootstrap.cc:492] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:18.771674 30970 tablet_bootstrap.cc:654] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.771811 30906 tablet_bootstrap.cc:492] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:18.772084 30906 tablet_bootstrap.cc:654] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.772444 30970 tablet_bootstrap.cc:492] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:18.772490 30970 ts_tablet_manager.cc:1397] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:18.772603 30970 raft_consensus.cc:357] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.772651 30970 raft_consensus.cc:383] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.772687 30970 raft_consensus.cc:738] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.772729 30970 consensus_queue.cc:260] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.772810 30970 ts_tablet_manager.cc:1428] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.772864 30970 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:18.773280 30970 tablet_bootstrap.cc:654] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.773321 30907 tablet_bootstrap.cc:492] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:18.773375 30907 ts_tablet_manager.cc:1397] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250112 21:06:18.773531 30907 raft_consensus.cc:357] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.773608 30907 raft_consensus.cc:383] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.773636 30907 raft_consensus.cc:738] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.773691 30907 consensus_queue.cc:260] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.773787 30907 ts_tablet_manager.cc:1428] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.773833 30907 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:18.773859 30906 tablet_bootstrap.cc:492] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:18.773900 30906 ts_tablet_manager.cc:1397] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:18.774008 30970 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:18.774045 30970 ts_tablet_manager.cc:1397] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:18.774152 30906 raft_consensus.cc:357] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.774173 30970 raft_consensus.cc:357] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.774209 30907 tablet_bootstrap.cc:654] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:18.774214 30906 raft_consensus.cc:383] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.774228 30970 raft_consensus.cc:383] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.774240 30906 raft_consensus.cc:738] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.774252 30970 raft_consensus.cc:738] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.774287 30906 consensus_queue.cc:260] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.774302 30970 consensus_queue.cc:260] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.774382 30906 ts_tablet_manager.cc:1428] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:18.774397 30970 ts_tablet_manager.cc:1428] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.774989 30907 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:18.775040 30907 ts_tablet_manager.cc:1397] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:18.775194 30907 raft_consensus.cc:357] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.775249 30907 raft_consensus.cc:383] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:18.775275 30907 raft_consensus.cc:738] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:18.775343 30907 consensus_queue.cc:260] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.775437 30907 ts_tablet_manager.cc:1428] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:18.922403 30910 raft_consensus.cc:491] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:18.922489 30910 raft_consensus.cc:513] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.922628 30910 leader_election.cc:290] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:18.925275 30283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd61ff11a25e482182da04059a77ed36" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c" is_pre_election: true
I20250112 21:06:18.925377 30283 raft_consensus.cc:2463] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 0.
I20250112 21:06:18.925514 30085 leader_election.cc:304] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:18.925622 30910 raft_consensus.cc:2798] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:18.925654 30910 raft_consensus.cc:491] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:18.925669 30910 raft_consensus.cc:3054] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.982146 30921 raft_consensus.cc:491] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:18.982234 30921 raft_consensus.cc:513] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.982389 30921 leader_election.cc:290] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:18.984695 30283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ad09a23ec4424187345c4914d19ac4" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c" is_pre_election: true
I20250112 21:06:18.984778 30283 raft_consensus.cc:2463] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 0.
I20250112 21:06:18.984917 29954 leader_election.cc:304] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.985049 30921 raft_consensus.cc:2798] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:18.985105 30921 raft_consensus.cc:491] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:18.985127 30921 raft_consensus.cc:3054] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.985620 30910 raft_consensus.cc:513] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.985757 30910 leader_election.cc:290] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:18.985780 30921 raft_consensus.cc:513] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.985883 30921 leader_election.cc:290] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:18.986011 30283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd61ff11a25e482182da04059a77ed36" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:18.986019 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ad09a23ec4424187345c4914d19ac4" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:18.986088 30283 raft_consensus.cc:3054] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.986115 30282 raft_consensus.cc:3054] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:18.986675 30283 raft_consensus.cc:2463] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 1.
I20250112 21:06:18.986840 30924 raft_consensus.cc:491] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:18.986871 30085 leader_election.cc:304] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:18.986884 30924 raft_consensus.cc:513] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.986963 30910 raft_consensus.cc:2798] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:18.986999 30910 raft_consensus.cc:695] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Leader. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:18.986999 30924 leader_election.cc:290] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:18.987020 30282 raft_consensus.cc:2463] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 1.
I20250112 21:06:18.987052 30910 consensus_queue.cc:237] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:18.987339 29954 leader_election.cc:304] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.987432 30921 raft_consensus.cc:2798] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:18.987479 30921 raft_consensus.cc:695] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Becoming Leader. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Running, Role: LEADER
I20250112 21:06:18.987517 30921 consensus_queue.cc:237] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:18.987913 29876 catalog_manager.cc:5526] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66). New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } }
I20250112 21:06:18.987975 29872 catalog_manager.cc:5526] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbe373da5cea408ea5c635c7f0164873 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } }
I20250112 21:06:18.990589 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dde2209dbc4c9a9a874f32baff5c5d" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873" is_pre_election: true
I20250112 21:06:18.990654 30021 raft_consensus.cc:2463] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 0.
I20250112 21:06:18.991058 30219 leader_election.cc:304] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:18.991151 30924 raft_consensus.cc:2798] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:18.991206 30924 raft_consensus.cc:491] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:18.991222 30924 raft_consensus.cc:3054] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.085112 30924 raft_consensus.cc:513] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:19.085295 30924 leader_election.cc:290] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Requested vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:19.085665 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dde2209dbc4c9a9a874f32baff5c5d" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873"
I20250112 21:06:19.085750 30021 raft_consensus.cc:3054] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.086303 30021 raft_consensus.cc:2463] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 1.
I20250112 21:06:19.086488 30219 leader_election.cc:304] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:19.086587 30924 raft_consensus.cc:2798] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:19.086750 30924 raft_consensus.cc:695] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Becoming Leader. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:19.086822 30924 consensus_queue.cc:237] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:19.087432 29876 catalog_manager.cc:5526] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c reported cstate change: term changed from 0 to 1, leader changed from <none> to b26f8d7d861f4663bd853aaec02b192c (127.26.82.67). New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } }
W20250112 21:06:19.196193 30983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:19.196378 30983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:19.197644 30983 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20250112 21:06:19.232954 30282 raft_consensus.cc:1270] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:19.233433 30921 consensus_queue.cc:1035] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:19.234634 30283 raft_consensus.cc:1270] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:19.235095 30021 raft_consensus.cc:1270] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:19.235378 30910 consensus_queue.cc:1035] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:19.235484 30981 consensus_queue.cc:1035] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:19.238415 31013 mvcc.cc:204] Tried to move back new op lower bound from 7113588650943131648 to 7113588649931104256. Current Snapshot: MvccSnapshot[applied={T|T < 7113588650943131648}]
INSERT report
rows total: 2000
time total: 27.4189 ms
time per row: 0.0137095 ms
I20250112 21:06:19.248818 29877 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56216:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:19.250190 30130 tablet_service.cc:1467] Processing CreateTablet for tablet d618a03030d6418baceccf148a9dfe57 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.250260 30261 tablet_service.cc:1467] Processing CreateTablet for tablet c650f14b4daa4c6682da63dec3e3ad6f (DEFAULT_TABLE table=rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.250277 29999 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.250337 30130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d618a03030d6418baceccf148a9dfe57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.250378 29999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.250378 30261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c650f14b4daa4c6682da63dec3e3ad6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.251549 30907 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:19.251562 30262 tablet_service.cc:1467] Processing CreateTablet for tablet d618a03030d6418baceccf148a9dfe57 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.251641 30262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d618a03030d6418baceccf148a9dfe57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.251698 30970 tablet_bootstrap.cc:492] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:19.251957 30907 tablet_bootstrap.cc:654] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.252035 30970 tablet_bootstrap.cc:654] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.252295 30906 tablet_bootstrap.cc:492] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:19.252511 30907 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:19.252549 30907 ts_tablet_manager.cc:1397] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:19.252611 30906 tablet_bootstrap.cc:654] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.252703 30907 raft_consensus.cc:357] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:19.252765 30907 raft_consensus.cc:383] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.252789 30907 raft_consensus.cc:738] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.252858 30907 consensus_queue.cc:260] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:19.250190 30132 tablet_service.cc:1467] Processing CreateTablet for tablet c650f14b4daa4c6682da63dec3e3ad6f (DEFAULT_TABLE table=rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.252959 30907 ts_tablet_manager.cc:1428] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.253002 30132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c650f14b4daa4c6682da63dec3e3ad6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.254238 30132 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.254338 30970 tablet_bootstrap.cc:492] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:19.254340 30132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.254385 30970 ts_tablet_manager.cc:1397] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:19.254516 30970 raft_consensus.cc:357] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.254563 30970 raft_consensus.cc:383] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.254587 30970 raft_consensus.cc:738] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.254637 30970 consensus_queue.cc:260] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.254741 30970 ts_tablet_manager.cc:1428] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:19.254792 30970 tablet_bootstrap.cc:492] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:19.255113 30970 tablet_bootstrap.cc:654] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.255613 30906 tablet_bootstrap.cc:492] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:19.255657 30906 ts_tablet_manager.cc:1397] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:19.255656 30970 tablet_bootstrap.cc:492] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:19.255730 30970 ts_tablet_manager.cc:1397] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:19.255800 30906 raft_consensus.cc:357] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.255862 30906 raft_consensus.cc:383] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.255885 30906 raft_consensus.cc:738] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.255874 30970 raft_consensus.cc:357] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.255934 30970 raft_consensus.cc:383] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.255928 30906 consensus_queue.cc:260] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.255960 30970 raft_consensus.cc:738] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.255999 30970 consensus_queue.cc:260] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.256036 30906 ts_tablet_manager.cc:1428] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.256083 30906 tablet_bootstrap.cc:492] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:19.256083 30970 ts_tablet_manager.cc:1428] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.256451 30906 tablet_bootstrap.cc:654] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.257119 30906 tablet_bootstrap.cc:492] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:19.257158 30906 ts_tablet_manager.cc:1397] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:19.257269 30906 raft_consensus.cc:357] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.257313 30906 raft_consensus.cc:383] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.257325 30906 raft_consensus.cc:738] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.257372 30906 consensus_queue.cc:260] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.257462 30906 ts_tablet_manager.cc:1428] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.257524 30906 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:19.257854 30906 tablet_bootstrap.cc:654] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.258338 30906 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:19.258374 30906 ts_tablet_manager.cc:1397] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:19.258464 30906 raft_consensus.cc:357] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:19.258502 30906 raft_consensus.cc:383] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.258515 30906 raft_consensus.cc:738] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.258553 30906 consensus_queue.cc:260] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:19.258641 30906 ts_tablet_manager.cc:1428] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.332202 30911 raft_consensus.cc:491] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:19.332315 30911 raft_consensus.cc:513] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:19.332448 30911 leader_election.cc:290] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:19.332688 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba4a23fe43d4e11a9462e8f5edca3e0" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c" is_pre_election: true
I20250112 21:06:19.332764 30152 raft_consensus.cc:2463] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 0.
I20250112 21:06:19.332952 29956 leader_election.cc:304] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:19.333035 30911 raft_consensus.cc:2798] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:19.333081 30911 raft_consensus.cc:491] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:19.333101 30911 raft_consensus.cc:3054] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.337100 30917 raft_consensus.cc:491] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:19.337163 30917 raft_consensus.cc:513] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.337263 30917 leader_election.cc:290] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:19.337447 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d618a03030d6418baceccf148a9dfe57" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c" is_pre_election: true
I20250112 21:06:19.337551 30282 raft_consensus.cc:2463] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 0.
I20250112 21:06:19.337739 30085 leader_election.cc:304] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:19.337828 30917 raft_consensus.cc:2798] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:19.337865 30917 raft_consensus.cc:491] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:19.337893 30917 raft_consensus.cc:3054] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.390383 30911 raft_consensus.cc:513] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:19.390389 30917 raft_consensus.cc:513] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.390532 30911 leader_election.cc:290] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:19.390533 30917 leader_election.cc:290] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:19.390733 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba4a23fe43d4e11a9462e8f5edca3e0" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:19.390730 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d618a03030d6418baceccf148a9dfe57" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:19.390810 30282 raft_consensus.cc:3054] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.390802 30152 raft_consensus.cc:3054] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.391343 30152 raft_consensus.cc:2463] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 1.
I20250112 21:06:19.391502 29956 leader_election.cc:304] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:19.391608 30911 raft_consensus.cc:2798] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:19.391636 30282 raft_consensus.cc:2463] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 1.
I20250112 21:06:19.391654 30911 raft_consensus.cc:695] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Becoming Leader. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Running, Role: LEADER
I20250112 21:06:19.391705 30911 consensus_queue.cc:237] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:19.391806 30085 leader_election.cc:304] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:19.391894 30917 raft_consensus.cc:2798] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:19.391933 30917 raft_consensus.cc:695] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Leader. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:19.391973 30917 consensus_queue.cc:237] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.392196 29872 catalog_manager.cc:5526] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbe373da5cea408ea5c635c7f0164873 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } }
I20250112 21:06:19.392580 29877 catalog_manager.cc:5526] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66). New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:19.552361 30910 raft_consensus.cc:491] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:19.552475 30910 raft_consensus.cc:513] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.552623 30910 leader_election.cc:290] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:19.552896 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c650f14b4daa4c6682da63dec3e3ad6f" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c" is_pre_election: true
I20250112 21:06:19.553004 30282 raft_consensus.cc:2463] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 0.
I20250112 21:06:19.553169 30085 leader_election.cc:304] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:19.553277 30910 raft_consensus.cc:2798] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:19.553318 30910 raft_consensus.cc:491] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:19.553339 30910 raft_consensus.cc:3054] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.592576 30910 raft_consensus.cc:513] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.592749 30910 leader_election.cc:290] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:19.592974 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c650f14b4daa4c6682da63dec3e3ad6f" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:19.593087 30282 raft_consensus.cc:3054] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.593858 30282 raft_consensus.cc:2463] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 1.
I20250112 21:06:19.594045 30085 leader_election.cc:304] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:19.594146 30910 raft_consensus.cc:2798] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:19.594229 30910 raft_consensus.cc:695] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Leader. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:19.594295 30910 consensus_queue.cc:237] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.594853 29877 catalog_manager.cc:5526] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66). New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
W20250112 21:06:19.680425 31031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:19.680575 31031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:19.681888 31031 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20250112 21:06:19.712764 30282 raft_consensus.cc:1270] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:19.713109 30910 consensus_queue.cc:1035] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:19.713639 30152 raft_consensus.cc:1270] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:19.713830 30911 consensus_queue.cc:1035] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:19.714572 30283 raft_consensus.cc:1270] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:19.714867 30910 consensus_queue.cc:1035] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 21.0489 ms
time per row: 0.0105245 ms
I20250112 21:06:19.726473 29876 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56216:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:19.727771 29999 tablet_service.cc:1467] Processing CreateTablet for tablet 6cb3ae487cbe46d58ceecc898e2ff13f (DEFAULT_TABLE table=rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.727818 30001 tablet_service.cc:1467] Processing CreateTablet for tablet fdb28e2f5c044d70bc87f9654504832c (DEFAULT_TABLE table=rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.727844 30262 tablet_service.cc:1467] Processing CreateTablet for tablet 6cb3ae487cbe46d58ceecc898e2ff13f (DEFAULT_TABLE table=rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.727890 30132 tablet_service.cc:1467] Processing CreateTablet for tablet b03a6dec8f774eaa83abfede3f9c84f5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.727917 29999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb3ae487cbe46d58ceecc898e2ff13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.727939 30262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb3ae487cbe46d58ceecc898e2ff13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.727982 30132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03a6dec8f774eaa83abfede3f9c84f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.728008 30001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb28e2f5c044d70bc87f9654504832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.729214 30907 tablet_bootstrap.cc:492] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:19.729223 30906 tablet_bootstrap.cc:492] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:19.729241 30261 tablet_service.cc:1467] Processing CreateTablet for tablet fdb28e2f5c044d70bc87f9654504832c (DEFAULT_TABLE table=rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.729327 30261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb28e2f5c044d70bc87f9654504832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.729610 30906 tablet_bootstrap.cc:654] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.729610 30907 tablet_bootstrap.cc:654] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.730055 30970 tablet_bootstrap.cc:492] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:19.730309 30906 tablet_bootstrap.cc:492] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:19.730351 30906 ts_tablet_manager.cc:1397] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:19.730456 30970 tablet_bootstrap.cc:654] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.730463 30001 tablet_service.cc:1467] Processing CreateTablet for tablet b03a6dec8f774eaa83abfede3f9c84f5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:19.730552 30001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03a6dec8f774eaa83abfede3f9c84f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:19.730573 30906 raft_consensus.cc:357] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:19.730633 30906 raft_consensus.cc:383] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.730654 30906 raft_consensus.cc:738] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.730693 30906 consensus_queue.cc:260] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:19.730777 30906 ts_tablet_manager.cc:1428] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.731001 30970 tablet_bootstrap.cc:492] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:19.731043 30970 ts_tablet_manager.cc:1397] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:19.731119 30907 tablet_bootstrap.cc:492] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:19.731153 30907 ts_tablet_manager.cc:1397] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:19.731187 30970 raft_consensus.cc:357] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.731242 30970 raft_consensus.cc:383] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.731262 30970 raft_consensus.cc:738] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.731281 30907 raft_consensus.cc:357] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.731307 30970 consensus_queue.cc:260] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.731348 30907 raft_consensus.cc:383] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.731369 30907 raft_consensus.cc:738] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.731410 30970 ts_tablet_manager.cc:1428] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.731410 30907 consensus_queue.cc:260] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.731470 30970 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:19.731511 30907 ts_tablet_manager.cc:1428] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:19.731572 30907 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:19.731902 30970 tablet_bootstrap.cc:654] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.731967 30907 tablet_bootstrap.cc:654] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.732699 30907 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:19.732748 30907 ts_tablet_manager.cc:1397] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:19.732892 30907 raft_consensus.cc:357] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.732946 30907 raft_consensus.cc:383] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.732970 30907 raft_consensus.cc:738] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.733016 30907 consensus_queue.cc:260] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.733122 30907 ts_tablet_manager.cc:1428] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.733170 30907 tablet_bootstrap.cc:492] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:19.733523 30907 tablet_bootstrap.cc:654] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:19.733543 30970 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:19.733580 30970 ts_tablet_manager.cc:1397] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:19.733708 30970 raft_consensus.cc:357] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.733765 30970 raft_consensus.cc:383] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.733785 30970 raft_consensus.cc:738] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.733837 30970 consensus_queue.cc:260] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.733928 30970 ts_tablet_manager.cc:1428] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.734124 30907 tablet_bootstrap.cc:492] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:19.734179 30907 ts_tablet_manager.cc:1397] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:19.734349 30907 raft_consensus.cc:357] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:19.734457 30907 raft_consensus.cc:383] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:19.734519 30907 raft_consensus.cc:738] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:19.734571 30907 consensus_queue.cc:260] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:19.734659 30907 ts_tablet_manager.cc:1428] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:19.743117 30981 raft_consensus.cc:491] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:19.743167 30981 raft_consensus.cc:513] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.743269 30981 leader_election.cc:290] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:19.743400 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cb3ae487cbe46d58ceecc898e2ff13f" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873" is_pre_election: true
I20250112 21:06:19.743459 30021 raft_consensus.cc:2463] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 0.
I20250112 21:06:19.743595 30219 leader_election.cc:304] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:19.743716 30981 raft_consensus.cc:2798] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:19.743772 30981 raft_consensus.cc:491] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:19.743793 30981 raft_consensus.cc:3054] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.790835 30981 raft_consensus.cc:513] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.790999 30981 leader_election.cc:290] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Requested vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:19.791203 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cb3ae487cbe46d58ceecc898e2ff13f" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873"
I20250112 21:06:19.791302 30021 raft_consensus.cc:3054] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.792173 30021 raft_consensus.cc:2463] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 1.
I20250112 21:06:19.792371 30219 leader_election.cc:304] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:19.792500 30981 raft_consensus.cc:2798] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:19.792623 30981 raft_consensus.cc:695] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Becoming Leader. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:19.792716 30981 consensus_queue.cc:237] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.793221 29876 catalog_manager.cc:5526] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c reported cstate change: term changed from 0 to 1, leader changed from <none> to b26f8d7d861f4663bd853aaec02b192c (127.26.82.67). New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:19.908496 30981 raft_consensus.cc:491] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:19.908584 30981 raft_consensus.cc:513] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.908756 30981 leader_election.cc:290] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:19.908931 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873" is_pre_election: true
I20250112 21:06:19.909024 30021 raft_consensus.cc:2463] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 0.
I20250112 21:06:19.909183 30219 leader_election.cc:304] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:19.909322 30981 raft_consensus.cc:2798] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:19.909369 30981 raft_consensus.cc:491] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:19.909390 30981 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.987174 30981 raft_consensus.cc:513] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.987331 30981 leader_election.cc:290] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Requested vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:19.987543 30021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe373da5cea408ea5c635c7f0164873"
I20250112 21:06:19.987648 30021 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:19.988514 30021 raft_consensus.cc:2463] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 1.
I20250112 21:06:19.988713 30219 leader_election.cc:304] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:19.988833 30981 raft_consensus.cc:2798] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:19.988878 30981 raft_consensus.cc:695] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Becoming Leader. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:19.988932 30981 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:19.989467 29876 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c reported cstate change: term changed from 0 to 1, leader changed from <none> to b26f8d7d861f4663bd853aaec02b192c (127.26.82.67). New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:20.009649 30911 raft_consensus.cc:491] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:20.009709 30911 raft_consensus.cc:513] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:20.009790 30911 leader_election.cc:290] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:20.009990 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b03a6dec8f774eaa83abfede3f9c84f5" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c" is_pre_election: true
I20250112 21:06:20.010094 30152 raft_consensus.cc:2463] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 0.
I20250112 21:06:20.010227 29956 leader_election.cc:304] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:20.010330 30911 raft_consensus.cc:2798] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:20.010377 30911 raft_consensus.cc:491] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:20.010392 30911 raft_consensus.cc:3054] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.088570 30911 raft_consensus.cc:513] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:20.088762 30911 leader_election.cc:290] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:20.088999 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b03a6dec8f774eaa83abfede3f9c84f5" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:20.089097 30152 raft_consensus.cc:3054] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.089919 30152 raft_consensus.cc:2463] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 1.
I20250112 21:06:20.090085 29956 leader_election.cc:304] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:20.090240 30921 raft_consensus.cc:2798] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:20.090297 30921 raft_consensus.cc:695] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Becoming Leader. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Running, Role: LEADER
I20250112 21:06:20.090353 30921 consensus_queue.cc:237] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } }
I20250112 21:06:20.090914 29876 catalog_manager.cc:5526] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbe373da5cea408ea5c635c7f0164873 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } }
I20250112 21:06:20.194612 30981 consensus_queue.cc:1035] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 21:06:20.245070 31076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:20.245219 31076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:20.246429 31076 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250112 21:06:20.255544 30216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.82.71:45067: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20250112 21:06:20.255888 30216 consensus_peers.cc:487] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Status: Network error: Client connection negotiation failed: client connection to 127.26.82.71:45067: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
Using table 'rebalance_test_table_3'
I20250112 21:06:20.274394 30020 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:20.274726 31070 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:20.274784 30152 raft_consensus.cc:1270] T b03a6dec8f774eaa83abfede3f9c84f5 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:20.275044 30921 consensus_queue.cc:1035] T b03a6dec8f774eaa83abfede3f9c84f5 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 17.8749 ms
time per row: 0.00893745 ms
I20250112 21:06:20.287192 29876 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56216:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:20.288565 30261 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8ccac51e3343d1b7d180d2fc09101f (DEFAULT_TABLE table=rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:20.288565 30130 tablet_service.cc:1467] Processing CreateTablet for tablet 284e96b0178c4ffd9842c70fbdd1658f (DEFAULT_TABLE table=rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:20.288609 30131 tablet_service.cc:1467] Processing CreateTablet for tablet a9f3456185e541c6abc00b7ec105a045 (DEFAULT_TABLE table=rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:20.288686 30262 tablet_service.cc:1467] Processing CreateTablet for tablet 284e96b0178c4ffd9842c70fbdd1658f (DEFAULT_TABLE table=rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:20.288707 30130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 284e96b0178c4ffd9842c70fbdd1658f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:20.288605 30132 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8ccac51e3343d1b7d180d2fc09101f (DEFAULT_TABLE table=rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:20.288869 30001 tablet_service.cc:1467] Processing CreateTablet for tablet a9f3456185e541c6abc00b7ec105a045 (DEFAULT_TABLE table=rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:20.288906 30132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8ccac51e3343d1b7d180d2fc09101f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:20.288952 30001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f3456185e541c6abc00b7ec105a045. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:20.289636 31120 tablet_bootstrap.cc:492] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:20.289937 31120 tablet_bootstrap.cc:654] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:20.288708 30261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8ccac51e3343d1b7d180d2fc09101f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:20.290097 30131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f3456185e541c6abc00b7ec105a045. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:20.290138 30262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 284e96b0178c4ffd9842c70fbdd1658f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:20.290995 31123 tablet_bootstrap.cc:492] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:20.291294 31123 tablet_bootstrap.cc:654] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:20.291466 31121 tablet_bootstrap.cc:492] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873: Bootstrap starting.
I20250112 21:06:20.291886 31121 tablet_bootstrap.cc:654] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:20.292043 31123 tablet_bootstrap.cc:492] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:20.292089 31120 tablet_bootstrap.cc:492] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:20.292124 31123 ts_tablet_manager.cc:1397] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:20.292140 31120 ts_tablet_manager.cc:1397] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:20.292253 31120 raft_consensus.cc:357] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.292255 31123 raft_consensus.cc:357] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.292299 31120 raft_consensus.cc:383] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:20.292313 31120 raft_consensus.cc:738] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:20.292316 31123 raft_consensus.cc:383] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:20.292340 31123 raft_consensus.cc:738] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:20.292352 31120 consensus_queue.cc:260] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.292389 31123 consensus_queue.cc:260] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.292447 31120 ts_tablet_manager.cc:1428] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.292471 31123 ts_tablet_manager.cc:1428] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.292505 31120 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:20.292526 31123 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:20.292852 31120 tablet_bootstrap.cc:654] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:20.292912 31123 tablet_bootstrap.cc:654] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:20.293177 31121 tablet_bootstrap.cc:492] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873: No bootstrap required, opened a new log
I20250112 21:06:20.293223 31121 ts_tablet_manager.cc:1397] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:20.293437 31121 raft_consensus.cc:357] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:20.293488 31121 raft_consensus.cc:383] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:20.293509 31121 raft_consensus.cc:738] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Initialized, Role: FOLLOWER
I20250112 21:06:20.293556 31121 consensus_queue.cc:260] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:20.293648 31121 ts_tablet_manager.cc:1428] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.293974 31120 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:20.294023 31120 ts_tablet_manager.cc:1397] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:20.294169 31120 raft_consensus.cc:357] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.294201 31123 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: No bootstrap required, opened a new log
I20250112 21:06:20.294215 31120 raft_consensus.cc:383] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:20.294234 31120 raft_consensus.cc:738] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:20.294240 31123 ts_tablet_manager.cc:1397] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:20.294263 31120 consensus_queue.cc:260] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.294353 31120 ts_tablet_manager.cc:1428] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.294364 31123 raft_consensus.cc:357] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.294399 31120 tablet_bootstrap.cc:492] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: Bootstrap starting.
I20250112 21:06:20.294412 31123 raft_consensus.cc:383] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:20.294431 31123 raft_consensus.cc:738] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:20.294462 31123 consensus_queue.cc:260] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.294541 31123 ts_tablet_manager.cc:1428] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.294721 31120 tablet_bootstrap.cc:654] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:20.295222 31120 tablet_bootstrap.cc:492] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: No bootstrap required, opened a new log
I20250112 21:06:20.295267 31120 ts_tablet_manager.cc:1397] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:20.295382 31120 raft_consensus.cc:357] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:20.295416 31120 raft_consensus.cc:383] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:20.295428 31120 raft_consensus.cc:738] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Initialized, Role: FOLLOWER
I20250112 21:06:20.295538 31120 consensus_queue.cc:260] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:20.295621 31120 ts_tablet_manager.cc:1428] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.347515 30921 raft_consensus.cc:491] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:20.347610 30921 raft_consensus.cc:513] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:20.347751 30921 leader_election.cc:290] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:20.347905 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f3456185e541c6abc00b7ec105a045" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c" is_pre_election: true
I20250112 21:06:20.347994 30152 raft_consensus.cc:2463] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 0.
I20250112 21:06:20.348193 29956 leader_election.cc:304] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:20.348294 30921 raft_consensus.cc:2798] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:20.348349 30921 raft_consensus.cc:491] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:20.348378 30921 raft_consensus.cc:3054] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.364533 30921 raft_consensus.cc:513] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:20.364647 30921 leader_election.cc:290] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:20.364857 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9f3456185e541c6abc00b7ec105a045" candidate_uuid: "cbe373da5cea408ea5c635c7f0164873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:20.364954 30152 raft_consensus.cc:3054] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.365773 30152 raft_consensus.cc:2463] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbe373da5cea408ea5c635c7f0164873 in term 1.
I20250112 21:06:20.365993 29956 leader_election.cc:304] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, cbe373da5cea408ea5c635c7f0164873; no voters:
I20250112 21:06:20.366103 30921 raft_consensus.cc:2798] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:20.366158 30921 raft_consensus.cc:695] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Becoming Leader. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Running, Role: LEADER
I20250112 21:06:20.366210 30921 consensus_queue.cc:237] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } }
I20250112 21:06:20.366739 29876 catalog_manager.cc:5526] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbe373da5cea408ea5c635c7f0164873 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } }
I20250112 21:06:20.387498 30917 raft_consensus.cc:491] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:20.387586 30917 raft_consensus.cc:513] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.387699 30917 leader_election.cc:290] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:20.387924 30283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "284e96b0178c4ffd9842c70fbdd1658f" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c" is_pre_election: true
I20250112 21:06:20.388000 30283 raft_consensus.cc:2463] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 0.
I20250112 21:06:20.388134 30085 leader_election.cc:304] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:20.388218 30917 raft_consensus.cc:2798] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:20.388257 30917 raft_consensus.cc:491] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:20.388278 30917 raft_consensus.cc:3054] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.397583 30910 raft_consensus.cc:491] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:20.397637 30910 raft_consensus.cc:513] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.397769 30910 leader_election.cc:290] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:20.397953 30283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8ccac51e3343d1b7d180d2fc09101f" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c" is_pre_election: true
I20250112 21:06:20.398025 30283 raft_consensus.cc:2463] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 0.
I20250112 21:06:20.398164 30085 leader_election.cc:304] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:20.398248 30910 raft_consensus.cc:2798] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:20.398291 30910 raft_consensus.cc:491] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:20.398322 30910 raft_consensus.cc:3054] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.466146 30917 raft_consensus.cc:513] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.466318 30917 leader_election.cc:290] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:20.466609 30283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "284e96b0178c4ffd9842c70fbdd1658f" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:20.466729 30283 raft_consensus.cc:3054] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.466806 30910 raft_consensus.cc:513] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.466929 30910 leader_election.cc:290] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:20.467164 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8ccac51e3343d1b7d180d2fc09101f" candidate_uuid: "04cea27f437b471d8557d2fe649a7a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:20.467262 30282 raft_consensus.cc:3054] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:20.467471 30283 raft_consensus.cc:2463] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 1.
I20250112 21:06:20.467631 30085 leader_election.cc:304] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:20.467741 30910 raft_consensus.cc:2798] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:20.467794 30910 raft_consensus.cc:695] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Leader. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:20.467837 30910 consensus_queue.cc:237] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.468052 30282 raft_consensus.cc:2463] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04cea27f437b471d8557d2fe649a7a7c in term 1.
I20250112 21:06:20.468286 30085 leader_election.cc:304] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:20.468374 30910 raft_consensus.cc:2798] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:20.468420 30910 raft_consensus.cc:695] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Leader. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:20.468425 29876 catalog_manager.cc:5526] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66). New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:20.468477 30910 consensus_queue.cc:237] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:20.469486 29876 catalog_manager.cc:5526] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66). New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
W20250112 21:06:20.592451 31132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:20.592590 31132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:20.593859 31132 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20250112 21:06:20.624146 30152 raft_consensus.cc:1270] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:20.625049 30282 raft_consensus.cc:1270] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:20.625134 30921 consensus_queue.cc:1035] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:20.625288 30910 consensus_queue.cc:1035] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:20.625865 30282 raft_consensus.cc:1270] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:20.626230 30910 consensus_queue.cc:1035] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 21.8929 ms
time per row: 0.0109464 ms
I20250112 21:06:20.639250 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.68:42459
--local_ip_for_outbound_sockets=127.26.82.68
--tserver_master_addrs=127.26.82.126:36745
--webserver_port=34025
--webserver_interface=127.26.82.68
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:20.713235 31165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:20.713372 31165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:20.713389 31165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:20.714588 31165 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:20.714635 31165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:20.714661 31165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.68
I20250112 21:06:20.716010 31165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.68:42459
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.82.68
--webserver_port=34025
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.31165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.68
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:20.716209 31165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:20.716400 31165 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:20.720743 31170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:20.721477 31171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:20.722193 31173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:20.723256 31165 server_base.cc:1034] running on GCE node
I20250112 21:06:20.723369 31165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:20.723541 31165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:20.724678 31165 hybrid_clock.cc:648] HybridClock initialized: now 1736715980724659 us; error 31 us; skew 500 ppm
I20250112 21:06:20.725747 31165 webserver.cc:458] Webserver started at http://127.26.82.68:34025/ using document root <none> and password file <none>
I20250112 21:06:20.725905 31165 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:20.725953 31165 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:20.727003 31165 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:20.727491 31179 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.727639 31165 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.727707 31165 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "142e8c9809704200abdf893a5d130c23"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:20.727952 31165 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:20.749485 31165 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:20.749722 31165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:20.749819 31165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:20.749998 31165 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:20.750284 31165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:20.750315 31165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.750346 31165 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:20.750377 31165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.755647 31165 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.68:42459
I20250112 21:06:20.755705 31292 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.68:42459 every 8 connection(s)
I20250112 21:06:20.755959 31165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250112 21:06:20.760083 31293 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:20.760151 31293 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:20.760277 31293 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:20.760598 29879 ts_manager.cc:194] Re-registered known tserver with Master: 142e8c9809704200abdf893a5d130c23 (127.26.82.68:42459)
I20250112 21:06:20.760946 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.68:41955
I20250112 21:06:20.763211 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 31165
I20250112 21:06:20.763465 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.69:41053
--local_ip_for_outbound_sockets=127.26.82.69
--tserver_master_addrs=127.26.82.126:36745
--webserver_port=44499
--webserver_interface=127.26.82.69
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:20.837524 31296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:20.837677 31296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:20.837723 31296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:20.838951 31296 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:20.838990 31296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:20.839023 31296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.69
I20250112 21:06:20.840238 31296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.69:41053
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.82.69
--webserver_port=44499
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.31296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.69
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:20.840446 31296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:20.840651 31296 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:20.844483 31301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:20.845358 31304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:20.846150 31302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:20.847023 31296 server_base.cc:1034] running on GCE node
I20250112 21:06:20.847183 31296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:20.847342 31296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:20.848459 31296 hybrid_clock.cc:648] HybridClock initialized: now 1736715980848451 us; error 26 us; skew 500 ppm
I20250112 21:06:20.849645 31296 webserver.cc:458] Webserver started at http://127.26.82.69:44499/ using document root <none> and password file <none>
I20250112 21:06:20.849844 31296 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:20.849946 31296 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:20.876260 31296 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:20.876833 31310 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.877023 31296 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:20.877189 31296 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "6e69a4b1e57242c2ba8e9e959c313e52"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:20.877578 31296 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:20.907361 31296 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:20.907608 31296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:20.907713 31296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:20.907917 31296 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:20.908191 31296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:20.908221 31296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.908254 31296 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:20.908279 31296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:20.913780 31296 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.69:41053
I20250112 21:06:20.913826 31423 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.69:41053 every 8 connection(s)
I20250112 21:06:20.914110 31296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250112 21:06:20.918386 31424 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:20.918466 31424 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:20.918639 31424 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:20.919013 29879 ts_manager.cc:194] Re-registered known tserver with Master: 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69:41053)
I20250112 21:06:20.919095 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 31296
I20250112 21:06:20.919361 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.70:36387
--local_ip_for_outbound_sockets=127.26.82.70
--tserver_master_addrs=127.26.82.126:36745
--webserver_port=40969
--webserver_interface=127.26.82.70
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:20.919504 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.69:54997
W20250112 21:06:20.993494 31427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:20.993633 31427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:20.993650 31427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:20.994788 31427 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:20.994832 31427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:20.994853 31427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.70
I20250112 21:06:20.996186 31427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.70:36387
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.82.70
--webserver_port=40969
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.31427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.70
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:20.996335 31427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:20.996498 31427 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:21.000926 31435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:21.001471 31433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:21.002118 31432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:21.003113 31427 server_base.cc:1034] running on GCE node
I20250112 21:06:21.003232 31427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:21.003400 31427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:21.004542 31427 hybrid_clock.cc:648] HybridClock initialized: now 1736715981004531 us; error 26 us; skew 500 ppm
I20250112 21:06:21.005699 31427 webserver.cc:458] Webserver started at http://127.26.82.70:40969/ using document root <none> and password file <none>
I20250112 21:06:21.005877 31427 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:21.005934 31427 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:21.007076 31427 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:21.007587 31441 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:21.007747 31427 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:21.007817 31427 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "b05fc98779d84f9589fe07913e1ba558"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:21.008073 31427 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:21.025857 31427 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:21.026113 31427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:21.026216 31427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:21.026386 31427 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:21.026650 31427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:21.026683 31427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:21.026712 31427 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:21.026737 31427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:21.032213 31427 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.70:36387
I20250112 21:06:21.032269 31554 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.70:36387 every 8 connection(s)
I20250112 21:06:21.032532 31427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20250112 21:06:21.033360 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 31427
I20250112 21:06:21.033632 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.71:45067
--local_ip_for_outbound_sockets=127.26.82.71
--tserver_master_addrs=127.26.82.126:36745
--webserver_port=44675
--webserver_interface=127.26.82.71
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:21.037295 31555 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:21.037391 31555 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:21.037604 31555 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:21.038121 29879 ts_manager.cc:194] Re-registered known tserver with Master: b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387)
I20250112 21:06:21.038645 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.70:53519
W20250112 21:06:21.108116 31558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:21.108263 31558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:21.108281 31558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:21.109486 31558 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:21.109524 31558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:21.109544 31558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.71
I20250112 21:06:21.110893 31558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.71:45067
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.82.71
--webserver_port=44675
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.31558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.71
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:21.111063 31558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:21.111261 31558 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:21.115375 31563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:21.116139 31566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:21.117151 31564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:21.118089 31558 server_base.cc:1034] running on GCE node
I20250112 21:06:21.118259 31558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:21.118441 31558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:21.119565 31558 hybrid_clock.cc:648] HybridClock initialized: now 1736715981119550 us; error 29 us; skew 500 ppm
I20250112 21:06:21.121364 31558 webserver.cc:458] Webserver started at http://127.26.82.71:44675/ using document root <none> and password file <none>
I20250112 21:06:21.121532 31558 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:21.121591 31558 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:21.122637 31558 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:21.123158 31572 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:21.123332 31558 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:21.123394 31558 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "aa93c8a23781463188b9ecba6c3bbf79"
format_stamp: "Formatted at 2025-01-12 21:06:14 on dist-test-slave-929r"
I20250112 21:06:21.123664 31558 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:21.137109 31558 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:21.137316 31558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:21.137395 31558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:21.137540 31558 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:21.137866 31579 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250112 21:06:21.138602 31558 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250112 21:06:21.138643 31558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:21.138679 31558 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250112 21:06:21.139156 31558 ts_tablet_manager.cc:610] Registered 1 tablets
I20250112 21:06:21.139186 31558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:21.139248 31579 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap starting.
I20250112 21:06:21.145308 31558 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.71:45067
I20250112 21:06:21.145359 31686 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.71:45067 every 8 connection(s)
I20250112 21:06:21.145637 31558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20250112 21:06:21.147876 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 31558
I20250112 21:06:21.148036 26953 test_util.cc:274] Using random seed: -1289629819
I20250112 21:06:21.148123 31579 log.cc:826] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:21.148801 31579 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:21.149025 31579 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap complete.
I20250112 21:06:21.150169 31579 ts_tablet_manager.cc:1397] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20250112 21:06:21.150859 31687 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:21.150949 31687 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:21.151140 31687 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:21.151664 29879 ts_manager.cc:194] Re-registered known tserver with Master: aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:21.152069 29879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.71:40527
I20250112 21:06:21.152009 31579 raft_consensus.cc:357] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:21.152287 31579 raft_consensus.cc:738] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Initialized, Role: FOLLOWER
I20250112 21:06:21.152384 31579 consensus_queue.cc:260] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:21.152616 31579 ts_tablet_manager.cc:1428] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20250112 21:06:21.152904 31687 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:21.154783 26953 test_workload.cc:405] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20250112 21:06:21.154910 26953 test_util.cc:274] Using random seed: -1289622947
I20250112 21:06:21.160056 26953 test_workload.cc:405] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20250112 21:06:21.160171 26953 test_util.cc:274] Using random seed: -1289617688
I20250112 21:06:21.166190 26953 test_workload.cc:405] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20250112 21:06:21.166319 26953 test_util.cc:274] Using random seed: -1289611537
I20250112 21:06:21.173897 26953 test_workload.cc:405] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20250112 21:06:21.174006 26953 test_util.cc:274] Using random seed: -1289603851
I20250112 21:06:21.179620 26953 test_workload.cc:405] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20250112 21:06:21.399704 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:21.399868 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.406441 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:21.408433 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:21.408591 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.419287 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.419685 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.499603 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.499616 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.502128 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.505697 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:21.506978 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:21.516193 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:21.522361 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:21.563493 30020 consensus_queue.cc:237] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 1.18, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:21.564062 30152 raft_consensus.cc:1270] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20250112 21:06:21.564427 31772 consensus_queue.cc:1035] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20250112 21:06:21.565431 30921 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.19: config changed from index -1 to 19, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:21.565600 30152 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index -1 to 19, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
W20250112 21:06:21.577580 29954 consensus_peers.cc:487] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 -> Peer b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387): Couldn't send request to peer b05fc98779d84f9589fe07913e1ba558. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ba4a23fe43d4e11a9462e8f5edca3e0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:21.585198 29879 catalog_manager.cc:5526] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index -1 to 19, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:21.606241 30151 consensus_queue.cc:237] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 71, Committed index: 71, Last appended: 1.71, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:21.606626 30019 raft_consensus.cc:1270] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20250112 21:06:21.606897 31759 consensus_queue.cc:1035] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
I20250112 21:06:21.607434 31776 raft_consensus.cc:2949] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.72: config changed from index -1 to 72, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:21.607694 30019 raft_consensus.cc:2949] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index -1 to 72, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
W20250112 21:06:21.624787 30087 consensus_peers.cc:487] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c -> Peer 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69:41053): Couldn't send request to peer 6e69a4b1e57242c2ba8e9e959c313e52. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 879a5ea2638e4c8e8df85215be98a1b6. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:21.685441 29879 catalog_manager.cc:5526] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index -1 to 72, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:21.705060 30151 consensus_queue.cc:237] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 1.20, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:21.706683 30283 raft_consensus.cc:1270] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 20. Preceding OpId from leader: term: 1 index: 21. (index mismatch)
I20250112 21:06:21.706940 31774 consensus_queue.cc:1035] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20250112 21:06:21.708165 31774 raft_consensus.cc:2949] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.21: config changed from index -1 to 21, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
W20250112 21:06:21.709297 30087 consensus_peers.cc:487] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c -> Peer 142e8c9809704200abdf893a5d130c23 (127.26.82.68:42459): Couldn't send request to peer 142e8c9809704200abdf893a5d130c23. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd61ff11a25e482182da04059a77ed36. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:21.710007 30283 raft_consensus.cc:2949] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Committing config change with OpId 1.21: config changed from index -1 to 21, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:21.761648 31293 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:21.781901 29871 catalog_manager.cc:5526] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index -1 to 21, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:21.803129 30283 consensus_queue.cc:237] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 1.22, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:21.803541 30021 raft_consensus.cc:1270] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 21. Preceding OpId from leader: term: 1 index: 23. (index mismatch)
W20250112 21:06:21.803694 30216 consensus_peers.cc:487] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0dde2209dbc4c9a9a874f32baff5c5d. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:21.803769 31754 consensus_queue.cc:1035] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.000s
I20250112 21:06:21.804445 31109 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.23: config changed from index -1 to 23, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:21.804709 30021 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Committing config change with OpId 1.23: config changed from index -1 to 23, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:21.885999 29879 catalog_manager.cc:5526] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index -1 to 23, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:21.903671 30020 consensus_queue.cc:237] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 1.98, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:21.904207 30151 raft_consensus.cc:1270] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 97. Preceding OpId from leader: term: 1 index: 99. (index mismatch)
I20250112 21:06:21.904458 30921 consensus_queue.cc:1035] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 99, Last known committed idx: 97, Time since last communication: 0.000s
I20250112 21:06:21.906646 31772 raft_consensus.cc:2949] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.99: config changed from index -1 to 99, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:21.906922 30151 raft_consensus.cc:2949] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Committing config change with OpId 1.99: config changed from index -1 to 99, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
W20250112 21:06:21.908197 29956 consensus_peers.cc:487] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 -> Peer 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69:41053): Couldn't send request to peer 6e69a4b1e57242c2ba8e9e959c313e52. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9f3456185e541c6abc00b7ec105a045. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:21.920305 31424 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:21.990752 31945 ts_tablet_manager.cc:927] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: Initiating tablet copy from peer cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:21.990983 31945 tablet_copy_client.cc:323] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:34885
I20250112 21:06:21.991199 29871 catalog_manager.cc:5526] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index -1 to 99, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:21.995229 30041 tablet_copy_service.cc:140] P cbe373da5cea408ea5c635c7f0164873: Received BeginTabletCopySession request for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 from peer b05fc98779d84f9589fe07913e1ba558 ({username='slave'} at 127.26.82.70:50289)
I20250112 21:06:21.995319 30041 tablet_copy_service.cc:161] P cbe373da5cea408ea5c635c7f0164873: Beginning new tablet copy session on tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 from peer b05fc98779d84f9589fe07913e1ba558 at {username='slave'} at 127.26.82.70:50289: session id = b05fc98779d84f9589fe07913e1ba558-3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:21.996263 30041 tablet_copy_source_session.cc:215] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:21.997283 31945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.000308 31945 tablet_copy_client.cc:806] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.000447 31945 tablet_copy_client.cc:670] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.010609 31945 tablet_copy_client.cc:538] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.012720 31945 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: Bootstrap starting.
I20250112 21:06:22.015513 31945 log.cc:826] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:22.023687 31945 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=0} inserts{seen=707 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.023941 31945 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: Bootstrap complete.
I20250112 21:06:22.024063 31945 ts_tablet_manager.cc:1397] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20250112 21:06:22.025203 31945 raft_consensus.cc:357] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.025375 31945 raft_consensus.cc:738] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b05fc98779d84f9589fe07913e1ba558, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.025485 31945 consensus_queue.cc:260] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 1.35, Last appended by leader: 35, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.025720 31945 ts_tablet_manager.cc:1428] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558: Time spent starting tablet: real 0.002s user 0.003s sys 0.002s
I20250112 21:06:22.025869 31555 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:22.026041 30041 tablet_copy_service.cc:342] P cbe373da5cea408ea5c635c7f0164873: Request end of tablet copy session b05fc98779d84f9589fe07913e1ba558-3ba4a23fe43d4e11a9462e8f5edca3e0 received from {username='slave'} at 127.26.82.70:50289
I20250112 21:06:22.026103 30041 tablet_copy_service.cc:434] P cbe373da5cea408ea5c635c7f0164873: ending tablet copy session b05fc98779d84f9589fe07913e1ba558-3ba4a23fe43d4e11a9462e8f5edca3e0 on tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 with peer b05fc98779d84f9589fe07913e1ba558
I20250112 21:06:22.049437 30152 consensus_queue.cc:237] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 114, Committed index: 114, Last appended: 1.114, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.051055 30283 raft_consensus.cc:1270] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20250112 21:06:22.053192 31776 consensus_queue.cc:1035] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20250112 21:06:22.054819 31774 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.115: config changed from index -1 to 115, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
W20250112 21:06:22.055645 30085 consensus_peers.cc:487] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c -> Peer b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387): Couldn't send request to peer b05fc98779d84f9589fe07913e1ba558. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e8ccac51e3343d1b7d180d2fc09101f. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.056071 29871 catalog_manager.cc:5526] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index -1 to 115, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:22.056254 30280 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index -1 to 115, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:22.084298 30280 consensus_queue.cc:237] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 169, Committed index: 169, Last appended: 1.169, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 170 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.085028 30020 raft_consensus.cc:1270] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 169. Preceding OpId from leader: term: 1 index: 170. (index mismatch)
I20250112 21:06:22.085271 31109 consensus_queue.cc:1035] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 170, Last known committed idx: 169, Time since last communication: 0.000s
I20250112 21:06:22.085827 31109 raft_consensus.cc:2949] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.170: config changed from index -1 to 170, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 170 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:22.085979 30020 raft_consensus.cc:2949] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Committing config change with OpId 1.170: config changed from index -1 to 170, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 170 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:22.087308 29871 catalog_manager.cc:5526] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index -1 to 170, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 170 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
W20250112 21:06:22.094260 30218 consensus_peers.cc:487] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c -> Peer 142e8c9809704200abdf893a5d130c23 (127.26.82.68:42459): Couldn't send request to peer 142e8c9809704200abdf893a5d130c23. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6cb3ae487cbe46d58ceecc898e2ff13f. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.117647 30281 consensus_queue.cc:237] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.118273 31641 raft_consensus.cc:1270] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:22.118578 31768 consensus_queue.cc:1035] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 21:06:22.119612 31768 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.120007 31641 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
W20250112 21:06:22.121416 30218 consensus_peers.cc:487] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c -> Peer 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69:41053): Couldn't send request to peer 6e69a4b1e57242c2ba8e9e959c313e52. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba45544d32d54e369dd5bd0e379c0cb1. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.122169 29879 catalog_manager.cc:5526] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 reported cstate change: config changed from index -1 to 2, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.174690 30021 consensus_queue.cc:237] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 102, Committed index: 102, Last appended: 1.102, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.175416 30150 raft_consensus.cc:1270] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 102. Preceding OpId from leader: term: 1 index: 103. (index mismatch)
I20250112 21:06:22.176571 31772 consensus_queue.cc:1035] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 103, Last known committed idx: 102, Time since last communication: 0.000s
W20250112 21:06:22.178650 29956 consensus_peers.cc:487] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 -> Peer 142e8c9809704200abdf893a5d130c23 (127.26.82.68:42459): Couldn't send request to peer 142e8c9809704200abdf893a5d130c23. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f71bda1d7cfd4744825f7286fa592f2c. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.178820 31772 raft_consensus.cc:2949] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.103: config changed from index -1 to 103, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:22.180084 29879 catalog_manager.cc:5526] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index -1 to 103, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:22.180331 30151 raft_consensus.cc:2949] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Committing config change with OpId 1.103: config changed from index -1 to 103, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:22.180932 32016 ts_tablet_manager.cc:927] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:22.181336 32016 tablet_copy_client.cc:323] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:22.188597 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet 879a5ea2638e4c8e8df85215be98a1b6 from peer 6e69a4b1e57242c2ba8e9e959c313e52 ({username='slave'} at 127.26.82.69:35733)
I20250112 21:06:22.188692 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet 879a5ea2638e4c8e8df85215be98a1b6 from peer 6e69a4b1e57242c2ba8e9e959c313e52 at {username='slave'} at 127.26.82.69:35733: session id = 6e69a4b1e57242c2ba8e9e959c313e52-879a5ea2638e4c8e8df85215be98a1b6
I20250112 21:06:22.189478 30172 tablet_copy_source_session.cc:215] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.191311 32016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879a5ea2638e4c8e8df85215be98a1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.197853 32016 tablet_copy_client.cc:806] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.198032 32016 tablet_copy_client.cc:670] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.201004 32016 tablet_copy_client.cc:538] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.202320 32016 tablet_bootstrap.cc:492] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap starting.
I20250112 21:06:22.207643 32016 log.cc:826] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:22.216068 30150 consensus_queue.cc:237] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 57, Committed index: 57, Last appended: 1.57, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:22.217126 30280 raft_consensus.cc:1270] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 57. Preceding OpId from leader: term: 1 index: 58. (index mismatch)
I20250112 21:06:22.217883 30917 consensus_queue.cc:1035] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 58, Last known committed idx: 57, Time since last communication: 0.000s
I20250112 21:06:22.220209 30917 raft_consensus.cc:2949] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.58: config changed from index -1 to 58, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:22.221483 29879 catalog_manager.cc:5526] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index -1 to 58, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
W20250112 21:06:22.223352 30085 consensus_peers.cc:487] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d618a03030d6418baceccf148a9dfe57. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.224550 30280 raft_consensus.cc:2949] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Committing config change with OpId 1.58: config changed from index -1 to 58, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:22.227672 32016 tablet_bootstrap.cc:492] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap replayed 1/1 log segments. Stats: ops{read=98 overwritten=0 applied=98 ignored=0} inserts{seen=783 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.227959 32016 tablet_bootstrap.cc:492] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap complete.
I20250112 21:06:22.228099 32016 ts_tablet_manager.cc:1397] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.001s
I20250112 21:06:22.229583 32016 raft_consensus.cc:357] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.229730 32016 raft_consensus.cc:738] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e69a4b1e57242c2ba8e9e959c313e52, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.229847 32016 consensus_queue.cc:260] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 98, Last appended: 1.98, Last appended by leader: 98, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.230060 32016 ts_tablet_manager.cc:1428] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250112 21:06:22.230525 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-879a5ea2638e4c8e8df85215be98a1b6 received from {username='slave'} at 127.26.82.69:35733
I20250112 21:06:22.230579 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-879a5ea2638e4c8e8df85215be98a1b6 on tablet 879a5ea2638e4c8e8df85215be98a1b6 with peer 6e69a4b1e57242c2ba8e9e959c313e52
I20250112 21:06:22.249534 30150 consensus_queue.cc:237] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 94, Committed index: 94, Last appended: 1.95, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.250383 30021 raft_consensus.cc:1270] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 94. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20250112 21:06:22.250599 32005 consensus_queue.cc:1035] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 94, Time since last communication: 0.000s
W20250112 21:06:22.251271 30085 consensus_peers.cc:487] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c -> Peer b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387): Couldn't send request to peer b05fc98779d84f9589fe07913e1ba558. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58c99277feb2489aa208bb04c7136ba3. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.251361 32005 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.96: config changed from index -1 to 96, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:22.252580 30021 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index -1 to 96, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:22.252645 29879 catalog_manager.cc:5526] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index -1 to 96, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:22.292564 30019 consensus_queue.cc:237] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 99, Committed index: 99, Last appended: 1.99, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.293004 30283 raft_consensus.cc:1270] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 99. Preceding OpId from leader: term: 1 index: 100. (index mismatch)
I20250112 21:06:22.293640 31762 consensus_queue.cc:1035] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 100, Last known committed idx: 99, Time since last communication: 0.000s
W20250112 21:06:22.294509 29956 consensus_peers.cc:487] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 -> Peer 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69:41053): Couldn't send request to peer 6e69a4b1e57242c2ba8e9e959c313e52. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f2ad09a23ec4424187345c4914d19ac4. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.295246 30921 raft_consensus.cc:2949] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.100: config changed from index -1 to 100, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.295547 30282 raft_consensus.cc:2949] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Committing config change with OpId 1.100: config changed from index -1 to 100, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New config: { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.296568 29879 catalog_manager.cc:5526] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index -1 to 100, VOTER 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.312608 32077 ts_tablet_manager.cc:927] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: Initiating tablet copy from peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:22.312943 32077 tablet_copy_client.cc:323] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:39003
I20250112 21:06:22.315107 32081 ts_tablet_manager.cc:927] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:22.315440 32081 tablet_copy_client.cc:323] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:22.320776 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet fd61ff11a25e482182da04059a77ed36 from peer 142e8c9809704200abdf893a5d130c23 ({username='slave'} at 127.26.82.68:57461)
I20250112 21:06:22.320837 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet fd61ff11a25e482182da04059a77ed36 from peer 142e8c9809704200abdf893a5d130c23 at {username='slave'} at 127.26.82.68:57461: session id = 142e8c9809704200abdf893a5d130c23-fd61ff11a25e482182da04059a77ed36
I20250112 21:06:22.321483 30172 tablet_copy_source_session.cc:215] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.322674 32081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd61ff11a25e482182da04059a77ed36. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.322669 30303 tablet_copy_service.cc:140] P b26f8d7d861f4663bd853aaec02b192c: Received BeginTabletCopySession request for tablet d0dde2209dbc4c9a9a874f32baff5c5d from peer aa93c8a23781463188b9ecba6c3bbf79 ({username='slave'} at 127.26.82.71:47507)
I20250112 21:06:22.322738 30303 tablet_copy_service.cc:161] P b26f8d7d861f4663bd853aaec02b192c: Beginning new tablet copy session on tablet d0dde2209dbc4c9a9a874f32baff5c5d from peer aa93c8a23781463188b9ecba6c3bbf79 at {username='slave'} at 127.26.82.71:47507: session id = aa93c8a23781463188b9ecba6c3bbf79-d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:22.323406 30303 tablet_copy_source_session.cc:215] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.323971 32077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0dde2209dbc4c9a9a874f32baff5c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.326086 32081 tablet_copy_client.cc:806] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.326236 32081 tablet_copy_client.cc:670] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.326993 32077 tablet_copy_client.cc:806] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.327147 32077 tablet_copy_client.cc:670] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.327155 32081 tablet_copy_client.cc:538] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.330461 32081 tablet_bootstrap.cc:492] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: Bootstrap starting.
I20250112 21:06:22.336459 32081 log.cc:826] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:22.336495 32077 tablet_copy_client.cc:538] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.337311 32077 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap starting.
I20250112 21:06:22.357179 32081 tablet_bootstrap.cc:492] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: Bootstrap replayed 1/1 log segments. Stats: ops{read=89 overwritten=0 applied=89 ignored=0} inserts{seen=720 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.357481 32081 tablet_bootstrap.cc:492] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: Bootstrap complete.
I20250112 21:06:22.357625 32081 ts_tablet_manager.cc:1397] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: Time spent bootstrapping tablet: real 0.027s user 0.017s sys 0.000s
I20250112 21:06:22.362498 32081 raft_consensus.cc:357] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.362746 32081 raft_consensus.cc:738] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142e8c9809704200abdf893a5d130c23, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.363131 32081 consensus_queue.cc:260] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 89, Last appended: 1.89, Last appended by leader: 89, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.363440 32081 ts_tablet_manager.cc:1428] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20250112 21:06:22.365415 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session 142e8c9809704200abdf893a5d130c23-fd61ff11a25e482182da04059a77ed36 received from {username='slave'} at 127.26.82.68:57461
I20250112 21:06:22.365476 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session 142e8c9809704200abdf893a5d130c23-fd61ff11a25e482182da04059a77ed36 on tablet fd61ff11a25e482182da04059a77ed36 with peer 142e8c9809704200abdf893a5d130c23
I20250112 21:06:22.379420 32077 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap replayed 1/1 log segments. Stats: ops{read=91 overwritten=0 applied=91 ignored=0} inserts{seen=763 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.379714 32077 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap complete.
I20250112 21:06:22.382761 30283 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 208, Committed index: 208, Last appended: 1.208, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.383324 30020 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 208. Preceding OpId from leader: term: 1 index: 209. (index mismatch)
I20250112 21:06:22.383562 31997 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 209, Last known committed idx: 208, Time since last communication: 0.000s
I20250112 21:06:22.383930 32077 ts_tablet_manager.cc:1397] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.007s
I20250112 21:06:22.384104 32077 raft_consensus.cc:357] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:22.384189 32077 raft_consensus.cc:738] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.384263 32077 consensus_queue.cc:260] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 91, Last appended: 1.91, Last appended by leader: 91, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 23 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:22.384879 31754 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.209: config changed from index -1 to 209, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:22.384996 32077 ts_tablet_manager.cc:1428] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:22.385282 30303 tablet_copy_service.cc:342] P b26f8d7d861f4663bd853aaec02b192c: Request end of tablet copy session aa93c8a23781463188b9ecba6c3bbf79-d0dde2209dbc4c9a9a874f32baff5c5d received from {username='slave'} at 127.26.82.71:47507
I20250112 21:06:22.385330 30303 tablet_copy_service.cc:434] P b26f8d7d861f4663bd853aaec02b192c: ending tablet copy session aa93c8a23781463188b9ecba6c3bbf79-d0dde2209dbc4c9a9a874f32baff5c5d on tablet d0dde2209dbc4c9a9a874f32baff5c5d with peer aa93c8a23781463188b9ecba6c3bbf79
I20250112 21:06:22.386202 30019 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Committing config change with OpId 1.209: config changed from index -1 to 209, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:22.386296 29871 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index -1 to 209, VOTER b05fc98779d84f9589fe07913e1ba558 (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
W20250112 21:06:22.386670 30216 consensus_peers.cc:487] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c -> Peer b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387): Couldn't send request to peer b05fc98779d84f9589fe07913e1ba558. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fdb28e2f5c044d70bc87f9654504832c. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:22.437018 31945 ts_tablet_manager.cc:927] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:22.437206 31945 tablet_copy_client.cc:323] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:22.440706 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet 8e8ccac51e3343d1b7d180d2fc09101f from peer b05fc98779d84f9589fe07913e1ba558 ({username='slave'} at 127.26.82.70:43073)
I20250112 21:06:22.440768 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet 8e8ccac51e3343d1b7d180d2fc09101f from peer b05fc98779d84f9589fe07913e1ba558 at {username='slave'} at 127.26.82.70:43073: session id = b05fc98779d84f9589fe07913e1ba558-8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:22.441378 30172 tablet_copy_source_session.cc:215] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.441936 31945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8ccac51e3343d1b7d180d2fc09101f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.443300 31945 tablet_copy_client.cc:806] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.443409 31945 tablet_copy_client.cc:670] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.448316 31945 tablet_copy_client.cc:538] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.449091 31945 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: Bootstrap starting.
I20250112 21:06:22.458039 31509 raft_consensus.cc:1212] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.18->[1.19-1.87] Dedup: 1.35->[1.36-1.87]
I20250112 21:06:22.466696 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:22.467015 32016 ts_tablet_manager.cc:927] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: Initiating tablet copy from peer cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:22.467154 32016 tablet_copy_client.cc:323] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:34885
I20250112 21:06:22.469794 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.473275 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.476696 30041 tablet_copy_service.cc:140] P cbe373da5cea408ea5c635c7f0164873: Received BeginTabletCopySession request for tablet a9f3456185e541c6abc00b7ec105a045 from peer 6e69a4b1e57242c2ba8e9e959c313e52 ({username='slave'} at 127.26.82.69:35797)
I20250112 21:06:22.476760 30041 tablet_copy_service.cc:161] P cbe373da5cea408ea5c635c7f0164873: Beginning new tablet copy session on tablet a9f3456185e541c6abc00b7ec105a045 from peer 6e69a4b1e57242c2ba8e9e959c313e52 at {username='slave'} at 127.26.82.69:35797: session id = 6e69a4b1e57242c2ba8e9e959c313e52-a9f3456185e541c6abc00b7ec105a045
I20250112 21:06:22.477277 30041 tablet_copy_source_session.cc:215] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.477713 32016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f3456185e541c6abc00b7ec105a045. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.479136 32016 tablet_copy_client.cc:806] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.479259 32016 tablet_copy_client.cc:670] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.483505 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.493759 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:22.494947 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:22.503054 32016 tablet_copy_client.cc:538] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.503958 32016 tablet_bootstrap.cc:492] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap starting.
I20250112 21:06:22.520259 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.544008 32016 tablet_bootstrap.cc:492] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap replayed 1/1 log segments. Stats: ops{read=181 overwritten=0 applied=181 ignored=0} inserts{seen=853 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.544296 32016 tablet_bootstrap.cc:492] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap complete.
I20250112 21:06:22.544533 32016 ts_tablet_manager.cc:1397] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent bootstrapping tablet: real 0.041s user 0.014s sys 0.009s
I20250112 21:06:22.544828 32016 raft_consensus.cc:357] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.544924 32016 raft_consensus.cc:738] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e69a4b1e57242c2ba8e9e959c313e52, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.545011 32016 consensus_queue.cc:260] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 181, Last appended: 1.181, Last appended by leader: 181, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 99 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.545114 32016 ts_tablet_manager.cc:1428] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:22.546193 30041 tablet_copy_service.cc:342] P cbe373da5cea408ea5c635c7f0164873: Request end of tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-a9f3456185e541c6abc00b7ec105a045 received from {username='slave'} at 127.26.82.69:35797
I20250112 21:06:22.546257 30041 tablet_copy_service.cc:434] P cbe373da5cea408ea5c635c7f0164873: ending tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-a9f3456185e541c6abc00b7ec105a045 on tablet a9f3456185e541c6abc00b7ec105a045 with peer 6e69a4b1e57242c2ba8e9e959c313e52
I20250112 21:06:22.562798 31945 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: Bootstrap replayed 1/1 log segments. Stats: ops{read=164 overwritten=0 applied=164 ignored=0} inserts{seen=849 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.563042 31945 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: Bootstrap complete.
I20250112 21:06:22.563174 31945 ts_tablet_manager.cc:1397] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: Time spent bootstrapping tablet: real 0.114s user 0.005s sys 0.015s
I20250112 21:06:22.563323 31945 raft_consensus.cc:357] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.563480 31945 raft_consensus.cc:738] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b05fc98779d84f9589fe07913e1ba558, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.563555 31945 consensus_queue.cc:260] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 164, Last appended: 1.164, Last appended by leader: 164, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.563774 31945 ts_tablet_manager.cc:1428] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20250112 21:06:22.564193 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session b05fc98779d84f9589fe07913e1ba558-8e8ccac51e3343d1b7d180d2fc09101f received from {username='slave'} at 127.26.82.70:43073
I20250112 21:06:22.564247 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session b05fc98779d84f9589fe07913e1ba558-8e8ccac51e3343d1b7d180d2fc09101f on tablet 8e8ccac51e3343d1b7d180d2fc09101f with peer b05fc98779d84f9589fe07913e1ba558
I20250112 21:06:22.621387 32077 ts_tablet_manager.cc:927] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:22.621551 32077 tablet_copy_client.cc:323] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:22.625957 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet d618a03030d6418baceccf148a9dfe57 from peer aa93c8a23781463188b9ecba6c3bbf79 ({username='slave'} at 127.26.82.71:39711)
I20250112 21:06:22.626019 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet d618a03030d6418baceccf148a9dfe57 from peer aa93c8a23781463188b9ecba6c3bbf79 at {username='slave'} at 127.26.82.71:39711: session id = aa93c8a23781463188b9ecba6c3bbf79-d618a03030d6418baceccf148a9dfe57
I20250112 21:06:22.626523 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:22.626609 30172 tablet_copy_source_session.cc:215] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.627147 32077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d618a03030d6418baceccf148a9dfe57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.628502 32077 tablet_copy_client.cc:806] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.628614 32077 tablet_copy_client.cc:670] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.629267 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.632196 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.638945 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.642664 32077 tablet_copy_client.cc:538] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.643391 32077 tablet_bootstrap.cc:492] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap starting.
I20250112 21:06:22.646642 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:22.659588 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.666360 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:22.674007 32077 tablet_bootstrap.cc:492] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap replayed 1/1 log segments. Stats: ops{read=108 overwritten=0 applied=108 ignored=0} inserts{seen=739 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.674233 32077 tablet_bootstrap.cc:492] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap complete.
I20250112 21:06:22.674357 32077 ts_tablet_manager.cc:1397] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.002s
I20250112 21:06:22.674485 32077 raft_consensus.cc:357] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:22.674558 32077 raft_consensus.cc:738] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.674631 32077 consensus_queue.cc:260] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 108, Last appended: 1.108, Last appended by leader: 108, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 58 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:22.674994 32077 ts_tablet_manager.cc:1428] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:22.675206 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session aa93c8a23781463188b9ecba6c3bbf79-d618a03030d6418baceccf148a9dfe57 received from {username='slave'} at 127.26.82.71:39711
I20250112 21:06:22.675264 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session aa93c8a23781463188b9ecba6c3bbf79-d618a03030d6418baceccf148a9dfe57 on tablet d618a03030d6418baceccf148a9dfe57 with peer aa93c8a23781463188b9ecba6c3bbf79
I20250112 21:06:22.676486 32081 ts_tablet_manager.cc:927] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: Initiating tablet copy from peer cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:22.676621 32081 tablet_copy_client.cc:323] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:34885
I20250112 21:06:22.679975 30041 tablet_copy_service.cc:140] P cbe373da5cea408ea5c635c7f0164873: Received BeginTabletCopySession request for tablet f71bda1d7cfd4744825f7286fa592f2c from peer 142e8c9809704200abdf893a5d130c23 ({username='slave'} at 127.26.82.68:49921)
I20250112 21:06:22.680033 30041 tablet_copy_service.cc:161] P cbe373da5cea408ea5c635c7f0164873: Beginning new tablet copy session on tablet f71bda1d7cfd4744825f7286fa592f2c from peer 142e8c9809704200abdf893a5d130c23 at {username='slave'} at 127.26.82.68:49921: session id = 142e8c9809704200abdf893a5d130c23-f71bda1d7cfd4744825f7286fa592f2c
I20250112 21:06:22.680719 30041 tablet_copy_source_session.cc:215] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.681145 32081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71bda1d7cfd4744825f7286fa592f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.682526 32081 tablet_copy_client.cc:806] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.682639 32081 tablet_copy_client.cc:670] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.685739 32185 ts_tablet_manager.cc:927] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: Initiating tablet copy from peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:22.686028 32185 tablet_copy_client.cc:323] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:39003
I20250112 21:06:22.688263 32081 tablet_copy_client.cc:538] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.688968 32081 tablet_bootstrap.cc:492] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: Bootstrap starting.
I20250112 21:06:22.690516 30303 tablet_copy_service.cc:140] P b26f8d7d861f4663bd853aaec02b192c: Received BeginTabletCopySession request for tablet 6cb3ae487cbe46d58ceecc898e2ff13f from peer 142e8c9809704200abdf893a5d130c23 ({username='slave'} at 127.26.82.68:48749)
I20250112 21:06:22.690565 30303 tablet_copy_service.cc:161] P b26f8d7d861f4663bd853aaec02b192c: Beginning new tablet copy session on tablet 6cb3ae487cbe46d58ceecc898e2ff13f from peer 142e8c9809704200abdf893a5d130c23 at {username='slave'} at 127.26.82.68:48749: session id = 142e8c9809704200abdf893a5d130c23-6cb3ae487cbe46d58ceecc898e2ff13f
I20250112 21:06:22.691125 30303 tablet_copy_source_session.cc:215] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.694927 32185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb3ae487cbe46d58ceecc898e2ff13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.700778 32185 tablet_copy_client.cc:806] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.700901 32185 tablet_copy_client.cc:670] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.703243 32016 ts_tablet_manager.cc:927] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: Initiating tablet copy from peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:22.703384 32016 tablet_copy_client.cc:323] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:39003
I20250112 21:06:22.703647 32185 tablet_copy_client.cc:538] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.704324 32185 tablet_bootstrap.cc:492] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: Bootstrap starting.
I20250112 21:06:22.707065 30303 tablet_copy_service.cc:140] P b26f8d7d861f4663bd853aaec02b192c: Received BeginTabletCopySession request for tablet ba45544d32d54e369dd5bd0e379c0cb1 from peer 6e69a4b1e57242c2ba8e9e959c313e52 ({username='slave'} at 127.26.82.69:45167)
I20250112 21:06:22.707129 30303 tablet_copy_service.cc:161] P b26f8d7d861f4663bd853aaec02b192c: Beginning new tablet copy session on tablet ba45544d32d54e369dd5bd0e379c0cb1 from peer 6e69a4b1e57242c2ba8e9e959c313e52 at {username='slave'} at 127.26.82.69:45167: session id = 6e69a4b1e57242c2ba8e9e959c313e52-ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:22.707645 30303 tablet_copy_source_session.cc:215] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.707994 32016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba45544d32d54e369dd5bd0e379c0cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.709336 32016 tablet_copy_client.cc:806] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.709450 32016 tablet_copy_client.cc:670] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.718644 32016 tablet_copy_client.cc:538] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.719043 32081 tablet_bootstrap.cc:492] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: Bootstrap replayed 1/1 log segments. Stats: ops{read=162 overwritten=0 applied=162 ignored=0} inserts{seen=793 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.719321 32081 tablet_bootstrap.cc:492] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: Bootstrap complete.
I20250112 21:06:22.719419 32016 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap starting.
I20250112 21:06:22.719530 32081 ts_tablet_manager.cc:1397] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: Time spent bootstrapping tablet: real 0.031s user 0.019s sys 0.002s
I20250112 21:06:22.719699 32081 raft_consensus.cc:357] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.719835 32081 raft_consensus.cc:738] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142e8c9809704200abdf893a5d130c23, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.719959 32081 consensus_queue.cc:260] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 162, Last appended: 1.162, Last appended by leader: 162, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 103 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.720628 32081 ts_tablet_manager.cc:1428] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:22.721254 30041 tablet_copy_service.cc:342] P cbe373da5cea408ea5c635c7f0164873: Request end of tablet copy session 142e8c9809704200abdf893a5d130c23-f71bda1d7cfd4744825f7286fa592f2c received from {username='slave'} at 127.26.82.68:49921
I20250112 21:06:22.721315 30041 tablet_copy_service.cc:434] P cbe373da5cea408ea5c635c7f0164873: ending tablet copy session 142e8c9809704200abdf893a5d130c23-f71bda1d7cfd4744825f7286fa592f2c on tablet f71bda1d7cfd4744825f7286fa592f2c with peer 142e8c9809704200abdf893a5d130c23
I20250112 21:06:22.725329 32016 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.725553 32016 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap complete.
I20250112 21:06:22.725661 32016 ts_tablet_manager.cc:1397] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250112 21:06:22.725943 32016 raft_consensus.cc:357] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.726029 32016 raft_consensus.cc:738] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e69a4b1e57242c2ba8e9e959c313e52, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.726079 32016 consensus_queue.cc:260] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.726192 32016 ts_tablet_manager.cc:1428] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:22.726598 30303 tablet_copy_service.cc:342] P b26f8d7d861f4663bd853aaec02b192c: Request end of tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-ba45544d32d54e369dd5bd0e379c0cb1 received from {username='slave'} at 127.26.82.69:45167
I20250112 21:06:22.726650 30303 tablet_copy_service.cc:434] P b26f8d7d861f4663bd853aaec02b192c: ending tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-ba45544d32d54e369dd5bd0e379c0cb1 on tablet ba45544d32d54e369dd5bd0e379c0cb1 with peer 6e69a4b1e57242c2ba8e9e959c313e52
I20250112 21:06:22.735929 31247 raft_consensus.cc:1212] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.20->[1.21-1.138] Dedup: 1.89->[1.90-1.138]
I20250112 21:06:22.746502 32185 tablet_bootstrap.cc:492] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: Bootstrap replayed 1/1 log segments. Stats: ops{read=238 overwritten=0 applied=238 ignored=0} inserts{seen=923 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.746719 32185 tablet_bootstrap.cc:492] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: Bootstrap complete.
I20250112 21:06:22.746901 32185 ts_tablet_manager.cc:1397] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: Time spent bootstrapping tablet: real 0.043s user 0.016s sys 0.012s
I20250112 21:06:22.747037 32185 raft_consensus.cc:357] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 170 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.747098 32185 raft_consensus.cc:738] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142e8c9809704200abdf893a5d130c23, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.747268 32185 consensus_queue.cc:260] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 238, Last appended: 1.238, Last appended by leader: 238, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 170 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:22.747797 32185 ts_tablet_manager.cc:1428] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20250112 21:06:22.748559 30303 tablet_copy_service.cc:342] P b26f8d7d861f4663bd853aaec02b192c: Request end of tablet copy session 142e8c9809704200abdf893a5d130c23-6cb3ae487cbe46d58ceecc898e2ff13f received from {username='slave'} at 127.26.82.68:48749
I20250112 21:06:22.748621 30303 tablet_copy_service.cc:434] P b26f8d7d861f4663bd853aaec02b192c: ending tablet copy session 142e8c9809704200abdf893a5d130c23-6cb3ae487cbe46d58ceecc898e2ff13f on tablet 6cb3ae487cbe46d58ceecc898e2ff13f with peer 142e8c9809704200abdf893a5d130c23
I20250112 21:06:22.760510 31945 ts_tablet_manager.cc:927] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: Initiating tablet copy from peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:22.760670 31945 tablet_copy_client.cc:323] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:39003
I20250112 21:06:22.764554 30303 tablet_copy_service.cc:140] P b26f8d7d861f4663bd853aaec02b192c: Received BeginTabletCopySession request for tablet fdb28e2f5c044d70bc87f9654504832c from peer b05fc98779d84f9589fe07913e1ba558 ({username='slave'} at 127.26.82.70:52605)
I20250112 21:06:22.764609 30303 tablet_copy_service.cc:161] P b26f8d7d861f4663bd853aaec02b192c: Beginning new tablet copy session on tablet fdb28e2f5c044d70bc87f9654504832c from peer b05fc98779d84f9589fe07913e1ba558 at {username='slave'} at 127.26.82.70:52605: session id = b05fc98779d84f9589fe07913e1ba558-fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:22.765228 30303 tablet_copy_source_session.cc:215] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.766273 31945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb28e2f5c044d70bc87f9654504832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.767565 31945 tablet_copy_client.cc:806] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.767674 31945 tablet_copy_client.cc:670] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.770083 31945 tablet_copy_client.cc:538] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.770839 31945 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: Bootstrap starting.
I20250112 21:06:22.773778 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.775600 31378 raft_consensus.cc:1212] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.71->[1.72-1.161] Dedup: 1.98->[1.99-1.161]
I20250112 21:06:22.777268 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.790303 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.796281 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:22.796425 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:22.798700 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:22.802103 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.819582 31945 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: Bootstrap replayed 1/1 log segments. Stats: ops{read=239 overwritten=0 applied=238 ignored=0} inserts{seen=869 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250112 21:06:22.819872 31945 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: Bootstrap complete.
I20250112 21:06:22.820020 31945 ts_tablet_manager.cc:1397] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: Time spent bootstrapping tablet: real 0.049s user 0.020s sys 0.004s
I20250112 21:06:22.820163 31945 raft_consensus.cc:357] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.820345 31945 raft_consensus.cc:738] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b05fc98779d84f9589fe07913e1ba558, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.820423 31945 consensus_queue.cc:260] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 238, Last appended: 1.239, Last appended by leader: 239, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.820920 31945 ts_tablet_manager.cc:1428] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:22.821689 30303 tablet_copy_service.cc:342] P b26f8d7d861f4663bd853aaec02b192c: Request end of tablet copy session b05fc98779d84f9589fe07913e1ba558-fdb28e2f5c044d70bc87f9654504832c received from {username='slave'} at 127.26.82.70:52605
I20250112 21:06:22.821753 30303 tablet_copy_service.cc:434] P b26f8d7d861f4663bd853aaec02b192c: ending tablet copy session b05fc98779d84f9589fe07913e1ba558-fdb28e2f5c044d70bc87f9654504832c on tablet fdb28e2f5c044d70bc87f9654504832c with peer b05fc98779d84f9589fe07913e1ba558
I20250112 21:06:22.825239 31641 raft_consensus.cc:1212] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.22->[1.23-1.151] Dedup: 1.91->[1.92-1.151]
I20250112 21:06:22.831511 31945 ts_tablet_manager.cc:927] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:22.831704 31945 tablet_copy_client.cc:323] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:22.833907 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet 58c99277feb2489aa208bb04c7136ba3 from peer b05fc98779d84f9589fe07913e1ba558 ({username='slave'} at 127.26.82.70:43073)
I20250112 21:06:22.833964 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet 58c99277feb2489aa208bb04c7136ba3 from peer b05fc98779d84f9589fe07913e1ba558 at {username='slave'} at 127.26.82.70:43073: session id = b05fc98779d84f9589fe07913e1ba558-58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:22.834506 30172 tablet_copy_source_session.cc:215] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.836745 31945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58c99277feb2489aa208bb04c7136ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.839612 31945 tablet_copy_client.cc:806] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.839784 31945 tablet_copy_client.cc:670] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.843667 31945 tablet_copy_client.cc:538] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.844408 31945 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: Bootstrap starting.
I20250112 21:06:22.855273 31378 raft_consensus.cc:1212] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.98->[1.99-1.222] Dedup: 1.181->[1.182-1.222]
I20250112 21:06:22.871021 32016 ts_tablet_manager.cc:927] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: Initiating tablet copy from peer cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:22.871251 32016 tablet_copy_client.cc:323] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:34885
I20250112 21:06:22.871626 30041 tablet_copy_service.cc:140] P cbe373da5cea408ea5c635c7f0164873: Received BeginTabletCopySession request for tablet f2ad09a23ec4424187345c4914d19ac4 from peer 6e69a4b1e57242c2ba8e9e959c313e52 ({username='slave'} at 127.26.82.69:35797)
I20250112 21:06:22.871693 30041 tablet_copy_service.cc:161] P cbe373da5cea408ea5c635c7f0164873: Beginning new tablet copy session on tablet f2ad09a23ec4424187345c4914d19ac4 from peer 6e69a4b1e57242c2ba8e9e959c313e52 at {username='slave'} at 127.26.82.69:35797: session id = 6e69a4b1e57242c2ba8e9e959c313e52-f2ad09a23ec4424187345c4914d19ac4
I20250112 21:06:22.872339 30041 tablet_copy_source_session.cc:215] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:22.872830 32016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ad09a23ec4424187345c4914d19ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:22.874190 32016 tablet_copy_client.cc:806] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:22.874315 32016 tablet_copy_client.cc:670] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:22.875313 31945 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: Bootstrap replayed 1/1 log segments. Stats: ops{read=182 overwritten=0 applied=182 ignored=0} inserts{seen=854 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.875540 31945 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: Bootstrap complete.
I20250112 21:06:22.875775 31945 ts_tablet_manager.cc:1397] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.004s
I20250112 21:06:22.875916 31945 raft_consensus.cc:357] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.875981 31945 raft_consensus.cc:738] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b05fc98779d84f9589fe07913e1ba558, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.876067 31945 consensus_queue.cc:260] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 182, Last appended: 1.182, Last appended by leader: 182, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:22.876878 31945 ts_tablet_manager.cc:1428] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:22.878145 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session b05fc98779d84f9589fe07913e1ba558-58c99277feb2489aa208bb04c7136ba3 received from {username='slave'} at 127.26.82.70:43073
I20250112 21:06:22.878202 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session b05fc98779d84f9589fe07913e1ba558-58c99277feb2489aa208bb04c7136ba3 on tablet 58c99277feb2489aa208bb04c7136ba3 with peer b05fc98779d84f9589fe07913e1ba558
I20250112 21:06:22.880964 32016 tablet_copy_client.cc:538] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:22.881806 32016 tablet_bootstrap.cc:492] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap starting.
I20250112 21:06:22.902024 30152 consensus_queue.cc:237] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 176, Committed index: 176, Last appended: 1.177, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 178 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.902671 31378 raft_consensus.cc:1270] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 177. Preceding OpId from leader: term: 1 index: 178. (index mismatch)
I20250112 21:06:22.902930 31759 consensus_queue.cc:1035] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 178, Last known committed idx: 176, Time since last communication: 0.000s
I20250112 21:06:22.903709 32089 raft_consensus.cc:2949] T 879a5ea2638e4c8e8df85215be98a1b6 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.178: config changed from index 72 to 178, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 178 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.903810 31378 raft_consensus.cc:2949] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Committing config change with OpId 1.178: config changed from index 72 to 178, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 178 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.905054 32016 tablet_bootstrap.cc:492] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap replayed 1/1 log segments. Stats: ops{read=158 overwritten=0 applied=158 ignored=0} inserts{seen=843 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:22.905077 29870 catalog_manager.cc:5526] T 879a5ea2638e4c8e8df85215be98a1b6 P 6e69a4b1e57242c2ba8e9e959c313e52 reported cstate change: config changed from index 72 to 178, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 178 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:22.905284 32016 tablet_bootstrap.cc:492] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: Bootstrap complete.
I20250112 21:06:22.905428 32016 ts_tablet_manager.cc:1397] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.007s
I20250112 21:06:22.905652 32016 raft_consensus.cc:357] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.905723 32016 raft_consensus.cc:738] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e69a4b1e57242c2ba8e9e959c313e52, State: Initialized, Role: FOLLOWER
I20250112 21:06:22.905784 32016 consensus_queue.cc:260] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 158, Last appended: 1.158, Last appended by leader: 158, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 100 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:22.905923 32016 ts_tablet_manager.cc:1428] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:22.906174 30041 tablet_copy_service.cc:342] P cbe373da5cea408ea5c635c7f0164873: Request end of tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-f2ad09a23ec4424187345c4914d19ac4 received from {username='slave'} at 127.26.82.69:35797
I20250112 21:06:22.906234 30041 tablet_copy_service.cc:434] P cbe373da5cea408ea5c635c7f0164873: ending tablet copy session 6e69a4b1e57242c2ba8e9e959c313e52-f2ad09a23ec4424187345c4914d19ac4 on tablet f2ad09a23ec4424187345c4914d19ac4 with peer 6e69a4b1e57242c2ba8e9e959c313e52
I20250112 21:06:22.908816 30001 tablet_service.cc:1514] Processing DeleteTablet for tablet 879a5ea2638e4c8e8df85215be98a1b6 with delete_type TABLET_DATA_TOMBSTONED (TS cbe373da5cea408ea5c635c7f0164873 not found in new config with opid_index 178) from {username='slave'} at 127.0.0.1:37368
I20250112 21:06:22.909904 32276 tablet_replica.cc:331] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: stopping tablet replica
I20250112 21:06:22.910019 32276 raft_consensus.cc:2238] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:22.910081 32276 raft_consensus.cc:2267] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:22.914461 32276 ts_tablet_manager.cc:1905] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:22.916775 32276 ts_tablet_manager.cc:1918] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.176
I20250112 21:06:22.916848 32276 log.cc:1198] T 879a5ea2638e4c8e8df85215be98a1b6 P cbe373da5cea408ea5c635c7f0164873: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/879a5ea2638e4c8e8df85215be98a1b6
I20250112 21:06:22.917383 29866 catalog_manager.cc:4872] TS cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885): tablet 879a5ea2638e4c8e8df85215be98a1b6 (table rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]) successfully deleted
I20250112 21:06:22.970679 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.971278 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.976586 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:22.978425 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.984012 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:22.985761 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:22.996650 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:23.001302 31509 raft_consensus.cc:1212] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.114->[1.115-1.238] Dedup: 1.164->[1.165-1.238]
I20250112 21:06:23.041327 30021 consensus_queue.cc:237] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 107, Committed index: 107, Last appended: 1.107, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:23.042618 31509 raft_consensus.cc:1270] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20250112 21:06:23.042833 31770 consensus_queue.cc:1035] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 107, Time since last communication: 0.000s
I20250112 21:06:23.043586 31770 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.108: config changed from index 19 to 108, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:23.044906 29879 catalog_manager.cc:5526] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index 19 to 108, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:23.045814 30131 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 with delete_type TABLET_DATA_TOMBSTONED (TS 04cea27f437b471d8557d2fe649a7a7c not found in new config with opid_index 108) from {username='slave'} at 127.0.0.1:56302
I20250112 21:06:23.046092 32332 tablet_replica.cc:331] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: stopping tablet replica
I20250112 21:06:23.046130 31509 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 19 to 108, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:23.046197 32332 raft_consensus.cc:2238] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:23.046343 32332 raft_consensus.cc:2267] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:23.046866 32332 ts_tablet_manager.cc:1905] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:23.050473 32332 ts_tablet_manager.cc:1918] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.107
I20250112 21:06:23.050558 32332 log.cc:1198] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P 04cea27f437b471d8557d2fe649a7a7c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:23.051067 29865 catalog_manager.cc:4872] TS 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009): tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 (table rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]) successfully deleted
I20250112 21:06:23.076751 31641 raft_consensus.cc:1212] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.57->[1.58-1.116] Dedup: 1.108->[1.109-1.116]
I20250112 21:06:23.112120 31377 raft_consensus.cc:1212] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 21:06:23.149277 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:23.162084 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.163429 31247 raft_consensus.cc:1212] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.102->[1.103-1.233] Dedup: 1.162->[1.163-1.233]
I20250112 21:06:23.166301 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.171664 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.173480 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.178040 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:23.180455 31508 raft_consensus.cc:1212] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.208->[1.209-1.255] Dedup: 1.239->[1.240-1.255]
I20250112 21:06:23.194170 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:23.252188 31245 raft_consensus.cc:1212] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.169->[1.170-1.257] Dedup: 1.238->[1.239-1.257]
I20250112 21:06:23.267551 31377 raft_consensus.cc:1212] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.99->[1.100-1.222] Dedup: 1.158->[1.159-1.222]
I20250112 21:06:23.301350 31508 raft_consensus.cc:1212] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.95->[1.96-1.240] Dedup: 1.182->[1.183-1.240]
I20250112 21:06:23.323374 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.327579 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.327999 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:23.330529 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.332337 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:23.334744 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.347882 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:23.384802 30019 consensus_queue.cc:237] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 262, Committed index: 262, Last appended: 1.262, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:23.385241 31247 raft_consensus.cc:1270] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20250112 21:06:23.385438 32284 consensus_queue.cc:1035] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20250112 21:06:23.386006 32284 raft_consensus.cc:2949] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.263: config changed from index 103 to 263, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.387362 29870 catalog_manager.cc:5526] T f71bda1d7cfd4744825f7286fa592f2c P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index 103 to 263, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.388139 31245 raft_consensus.cc:2949] T f71bda1d7cfd4744825f7286fa592f2c P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 103 to 263, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.389282 30131 tablet_service.cc:1514] Processing DeleteTablet for tablet f71bda1d7cfd4744825f7286fa592f2c with delete_type TABLET_DATA_TOMBSTONED (TS 04cea27f437b471d8557d2fe649a7a7c not found in new config with opid_index 263) from {username='slave'} at 127.0.0.1:56302
I20250112 21:06:23.389421 32332 tablet_replica.cc:331] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: stopping tablet replica
I20250112 21:06:23.389513 32332 raft_consensus.cc:2238] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:23.389587 32332 raft_consensus.cc:2267] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:23.390035 32332 ts_tablet_manager.cc:1905] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:23.391532 32332 ts_tablet_manager.cc:1918] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.262
I20250112 21:06:23.391592 32332 log.cc:1198] T f71bda1d7cfd4744825f7286fa592f2c P 04cea27f437b471d8557d2fe649a7a7c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/f71bda1d7cfd4744825f7286fa592f2c
I20250112 21:06:23.392462 29865 catalog_manager.cc:4872] TS 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009): tablet f71bda1d7cfd4744825f7286fa592f2c (table rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]) successfully deleted
I20250112 21:06:23.460511 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.466094 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.467316 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:23.475340 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:23.476047 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.492313 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:23.496485 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.556180 30282 consensus_queue.cc:237] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 299, Committed index: 299, Last appended: 1.299, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:23.556569 31245 raft_consensus.cc:1270] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 299. Preceding OpId from leader: term: 1 index: 300. (index mismatch)
I20250112 21:06:23.556800 31754 consensus_queue.cc:1035] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 300, Last known committed idx: 298, Time since last communication: 0.000s
I20250112 21:06:23.557658 31754 raft_consensus.cc:2949] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.300: config changed from index 170 to 300, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.558698 31245 raft_consensus.cc:2949] T 6cb3ae487cbe46d58ceecc898e2ff13f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Committing config change with OpId 1.300: config changed from index 170 to 300, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.559026 29872 catalog_manager.cc:5526] T 6cb3ae487cbe46d58ceecc898e2ff13f P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 170 to 300, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 300 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.559931 30001 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cb3ae487cbe46d58ceecc898e2ff13f with delete_type TABLET_DATA_TOMBSTONED (TS cbe373da5cea408ea5c635c7f0164873 not found in new config with opid_index 300) from {username='slave'} at 127.0.0.1:37368
I20250112 21:06:23.561091 32490 tablet_replica.cc:331] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: stopping tablet replica
I20250112 21:06:23.561498 32490 raft_consensus.cc:2238] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:23.561563 32490 raft_consensus.cc:2267] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:23.563529 32490 ts_tablet_manager.cc:1905] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:23.569432 32490 ts_tablet_manager.cc:1918] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.299
I20250112 21:06:23.569489 32490 log.cc:1198] T 6cb3ae487cbe46d58ceecc898e2ff13f P cbe373da5cea408ea5c635c7f0164873: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/6cb3ae487cbe46d58ceecc898e2ff13f
I20250112 21:06:23.575148 29866 catalog_manager.cc:4872] TS cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885): tablet 6cb3ae487cbe46d58ceecc898e2ff13f (table rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]) successfully deleted
I20250112 21:06:23.642351 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:23.646713 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.647322 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.653507 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.653983 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:23.654964 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.661283 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:23.728833 30283 consensus_queue.cc:237] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 305, Committed index: 305, Last appended: 1.305, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 306 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:23.729267 31641 raft_consensus.cc:1270] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 305. Preceding OpId from leader: term: 1 index: 307. (index mismatch)
I20250112 21:06:23.729565 31778 consensus_queue.cc:1035] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 306, Last known committed idx: 305, Time since last communication: 0.000s
I20250112 21:06:23.730223 31997 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.306: config changed from index 23 to 306, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 306 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:23.731294 31641 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Committing config change with OpId 1.306: config changed from index 23 to 306, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 306 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:23.731392 29872 catalog_manager.cc:5526] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 23 to 306, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 306 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:23.732568 30001 tablet_service.cc:1514] Processing DeleteTablet for tablet d0dde2209dbc4c9a9a874f32baff5c5d with delete_type TABLET_DATA_TOMBSTONED (TS cbe373da5cea408ea5c635c7f0164873 not found in new config with opid_index 306) from {username='slave'} at 127.0.0.1:37368
I20250112 21:06:23.732690 32490 tablet_replica.cc:331] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: stopping tablet replica
I20250112 21:06:23.732740 32490 raft_consensus.cc:2238] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:23.732777 32490 raft_consensus.cc:2267] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:23.733243 32490 ts_tablet_manager.cc:1905] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:23.734638 32490 ts_tablet_manager.cc:1918] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.305
I20250112 21:06:23.734689 32490 log.cc:1198] T d0dde2209dbc4c9a9a874f32baff5c5d P cbe373da5cea408ea5c635c7f0164873: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:23.735234 29866 catalog_manager.cc:4872] TS cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885): tablet d0dde2209dbc4c9a9a874f32baff5c5d (table rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]) successfully deleted
I20250112 21:06:23.818874 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.820202 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.827153 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.831894 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:23.840633 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:23.843119 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:23.850354 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:23.921784 30151 consensus_queue.cc:237] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 310, Committed index: 310, Last appended: 1.310, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 311 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:23.922353 31245 raft_consensus.cc:1270] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 310. Preceding OpId from leader: term: 1 index: 311. (index mismatch)
I20250112 21:06:23.922729 31759 consensus_queue.cc:1035] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 311, Last known committed idx: 310, Time since last communication: 0.000s
I20250112 21:06:23.927430 32089 raft_consensus.cc:2949] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.311: config changed from index 21 to 311, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 311 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.927768 31247 raft_consensus.cc:2949] T fd61ff11a25e482182da04059a77ed36 P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Committing config change with OpId 1.311: config changed from index 21 to 311, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 311 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.928649 29871 catalog_manager.cc:5526] T fd61ff11a25e482182da04059a77ed36 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index 21 to 311, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 311 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:23.930001 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet fd61ff11a25e482182da04059a77ed36 with delete_type TABLET_DATA_TOMBSTONED (TS b26f8d7d861f4663bd853aaec02b192c not found in new config with opid_index 311) from {username='slave'} at 127.0.0.1:47298
I20250112 21:06:23.930248 32603 tablet_replica.cc:331] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: stopping tablet replica
I20250112 21:06:23.930516 32603 raft_consensus.cc:2238] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:23.930655 32603 raft_consensus.cc:2267] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:23.935161 32603 ts_tablet_manager.cc:1905] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:23.942989 32603 ts_tablet_manager.cc:1918] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.310
I20250112 21:06:23.943131 32603 log.cc:1198] T fd61ff11a25e482182da04059a77ed36 P b26f8d7d861f4663bd853aaec02b192c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/fd61ff11a25e482182da04059a77ed36
I20250112 21:06:23.944051 29863 catalog_manager.cc:4872] TS b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): tablet fd61ff11a25e482182da04059a77ed36 (table rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]) successfully deleted
I20250112 21:06:24.013419 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.014900 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.020426 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:24.022475 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:24.044869 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.063545 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.066309 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:24.113121 30020 consensus_queue.cc:237] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 298, Committed index: 298, Last appended: 1.298, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:24.113642 31377 raft_consensus.cc:1270] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 298. Preceding OpId from leader: term: 1 index: 299. (index mismatch)
I20250112 21:06:24.114015 30921 consensus_queue.cc:1035] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 299, Last known committed idx: 298, Time since last communication: 0.000s
I20250112 21:06:24.118244 31731 raft_consensus.cc:2949] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.299: config changed from index 99 to 299, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:24.118400 31377 raft_consensus.cc:2949] T a9f3456185e541c6abc00b7ec105a045 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Committing config change with OpId 1.299: config changed from index 99 to 299, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:24.119616 29873 catalog_manager.cc:5526] T a9f3456185e541c6abc00b7ec105a045 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index 99 to 299, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 299 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:24.120949 30131 tablet_service.cc:1514] Processing DeleteTablet for tablet a9f3456185e541c6abc00b7ec105a045 with delete_type TABLET_DATA_TOMBSTONED (TS 04cea27f437b471d8557d2fe649a7a7c not found in new config with opid_index 299) from {username='slave'} at 127.0.0.1:56302
I20250112 21:06:24.121253 32659 tablet_replica.cc:331] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: stopping tablet replica
I20250112 21:06:24.121438 32659 raft_consensus.cc:2238] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:24.121532 32659 raft_consensus.cc:2267] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:24.123435 32659 ts_tablet_manager.cc:1905] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:24.135098 32659 ts_tablet_manager.cc:1918] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.298
I20250112 21:06:24.135211 32659 log.cc:1198] T a9f3456185e541c6abc00b7ec105a045 P 04cea27f437b471d8557d2fe649a7a7c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/a9f3456185e541c6abc00b7ec105a045
I20250112 21:06:24.136127 29865 catalog_manager.cc:4872] TS 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009): tablet a9f3456185e541c6abc00b7ec105a045 (table rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]) successfully deleted
I20250112 21:06:24.201328 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.206379 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.213869 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:24.216765 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.218753 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.229214 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:24.253221 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:24.302762 30282 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 381, Committed index: 381, Last appended: 1.381, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 382 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:24.303118 31509 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 381. Preceding OpId from leader: term: 1 index: 382. (index mismatch)
I20250112 21:06:24.303296 31997 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 382, Last known committed idx: 381, Time since last communication: 0.000s
I20250112 21:06:24.305900 31997 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.382: config changed from index 209 to 382, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 382 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.307181 29870 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 209 to 382, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 382 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.307312 31507 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.382: config changed from index 209 to 382, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 382 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.308707 30001 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb28e2f5c044d70bc87f9654504832c with delete_type TABLET_DATA_TOMBSTONED (TS cbe373da5cea408ea5c635c7f0164873 not found in new config with opid_index 382) from {username='slave'} at 127.0.0.1:37368
I20250112 21:06:24.309510 32714 tablet_replica.cc:331] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: stopping tablet replica
I20250112 21:06:24.309813 32714 raft_consensus.cc:2238] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:24.309877 32714 raft_consensus.cc:2267] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:24.310357 32714 ts_tablet_manager.cc:1905] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:24.313184 32714 ts_tablet_manager.cc:1918] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.381
I20250112 21:06:24.313365 32714 log.cc:1198] T fdb28e2f5c044d70bc87f9654504832c P cbe373da5cea408ea5c635c7f0164873: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:24.314394 29866 catalog_manager.cc:4872] TS cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885): tablet fdb28e2f5c044d70bc87f9654504832c (table rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]) successfully deleted
I20250112 21:06:24.393723 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:24.397584 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.402460 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:24.408152 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.417026 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:24.429675 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.430267 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.465943 30282 consensus_queue.cc:237] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:24.466364 31377 raft_consensus.cc:1270] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 21:06:24.466643 31997 consensus_queue.cc:1035] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 21:06:24.467792 31997 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:24.468259 31378 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:24.469606 29872 catalog_manager.cc:5526] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 reported cstate change: config changed from index 2 to 3, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) evicted. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:24.474463 31621 tablet_service.cc:1514] Processing DeleteTablet for tablet ba45544d32d54e369dd5bd0e379c0cb1 with delete_type TABLET_DATA_TOMBSTONED (TS aa93c8a23781463188b9ecba6c3bbf79 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:36816
I20250112 21:06:24.475417 309 tablet_replica.cc:331] stopping tablet replica
I20250112 21:06:24.475571 309 raft_consensus.cc:2238] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:24.475653 309 raft_consensus.cc:2267] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:24.475980 309 ts_tablet_manager.cc:1905] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:24.479347 309 ts_tablet_manager.cc:1918] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20250112 21:06:24.479411 309 log.cc:1198] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:24.480129 29863 catalog_manager.cc:4872] TS aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): tablet ba45544d32d54e369dd5bd0e379c0cb1 (table TestTable [id=9982a2b4bec0443585ea28fb19554c5e]) successfully deleted
I20250112 21:06:24.545775 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.545892 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.554425 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.560690 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.568246 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:24.576272 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:24.585458 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:24.633040 30152 consensus_queue.cc:237] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 317, Committed index: 317, Last appended: 1.318, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:24.634006 31507 raft_consensus.cc:1270] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20250112 21:06:24.634624 31759 consensus_queue.cc:1035] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 317, Time since last communication: 0.000s
I20250112 21:06:24.635164 31759 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.319: config changed from index 115 to 319, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.635537 31507 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 115 to 319, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.636641 29870 catalog_manager.cc:5526] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index 115 to 319, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.638334 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8ccac51e3343d1b7d180d2fc09101f with delete_type TABLET_DATA_TOMBSTONED (TS b26f8d7d861f4663bd853aaec02b192c not found in new config with opid_index 319) from {username='slave'} at 127.0.0.1:47298
I20250112 21:06:24.640511 360 tablet_replica.cc:331] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: stopping tablet replica
I20250112 21:06:24.641319 360 raft_consensus.cc:2238] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:24.641377 360 pending_rounds.cc:62] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Trying to abort 1 pending ops.
I20250112 21:06:24.641433 360 pending_rounds.cc:69] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Aborting op as it isn't in flight: id { term: 1 index: 318 } timestamp: 7113588673050673152 op_type: WRITE_OP write_request { tablet_id: "8e8ccac51e3343d1b7d180d2fc09101f" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000HI\352(z\020\273`\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""b245b719fdf009d6" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7113588673039921152 authz_token { token_data: "\010\371\337\220\274\006\"3\n\005slave\022*\n 647ae10537334714afb53ce0d07b82c5\020\001\030\001 \001(\001" signature: "<redacted>""b%]\370>,\266\235A\376A}\007\005\330\374\r\315Z\203Xt\371\304\336\314}\260\321\233\212T\241\013\014\306et-u\234B\203\333\247\322\366\231\374\355&\205;\306+\247\227\266\030:\355dQ\037" signing_key_seq_num: 0 } } request_id { client_id: "d9485942424441a2be7c8b6968888695" seq_no: 918 first_incomplete_seq_no: 918 attempt_no: 0 }
I20250112 21:06:24.641782 360 raft_consensus.cc:2267] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:24.647415 360 ts_tablet_manager.cc:1905] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:24.652117 360 ts_tablet_manager.cc:1918] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.318
I20250112 21:06:24.652218 360 log.cc:1198] T 8e8ccac51e3343d1b7d180d2fc09101f P b26f8d7d861f4663bd853aaec02b192c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:24.652614 29863 catalog_manager.cc:4872] TS b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): tablet 8e8ccac51e3343d1b7d180d2fc09101f (table rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]) successfully deleted
I20250112 21:06:24.894301 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.899430 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.903046 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.903157 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:24.907384 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:24.911090 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:24.917805 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:24.951005 30152 consensus_queue.cc:237] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 403, Committed index: 403, Last appended: 1.403, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 404 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } }
I20250112 21:06:24.951654 31509 raft_consensus.cc:1270] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 403. Preceding OpId from leader: term: 1 index: 404. (index mismatch)
I20250112 21:06:24.951957 31710 consensus_queue.cc:1035] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 404, Last known committed idx: 403, Time since last communication: 0.000s
I20250112 21:06:24.952646 31774 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.404: config changed from index 96 to 404, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 404 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.953675 31509 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.404: config changed from index 96 to 404, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 404 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.953794 29872 catalog_manager.cc:5526] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index 96 to 404, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 404 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } }
I20250112 21:06:24.954922 30001 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c99277feb2489aa208bb04c7136ba3 with delete_type TABLET_DATA_TOMBSTONED (TS cbe373da5cea408ea5c635c7f0164873 not found in new config with opid_index 404) from {username='slave'} at 127.0.0.1:37368
I20250112 21:06:24.955130 415 tablet_replica.cc:331] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: stopping tablet replica
I20250112 21:06:24.955188 415 raft_consensus.cc:2238] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:24.955233 415 raft_consensus.cc:2267] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:24.956475 415 ts_tablet_manager.cc:1905] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:24.958855 415 ts_tablet_manager.cc:1918] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.403
I20250112 21:06:24.958956 415 log.cc:1198] T 58c99277feb2489aa208bb04c7136ba3 P cbe373da5cea408ea5c635c7f0164873: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:24.959553 29866 catalog_manager.cc:4872] TS cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885): tablet 58c99277feb2489aa208bb04c7136ba3 (table rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]) successfully deleted
I20250112 21:06:25.003361 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.006619 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.007994 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.008895 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:25.012351 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:25.017359 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.023592 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:25.057718 30152 consensus_queue.cc:237] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 326, Committed index: 326, Last appended: 1.326, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 327 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:25.058089 31640 raft_consensus.cc:1270] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 326. Preceding OpId from leader: term: 1 index: 327. (index mismatch)
I20250112 21:06:25.058326 31759 consensus_queue.cc:1035] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 327, Last known committed idx: 326, Time since last communication: 0.000s
I20250112 21:06:25.058893 31759 raft_consensus.cc:2949] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.327: config changed from index 58 to 327, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 327 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.059151 31640 raft_consensus.cc:2949] T d618a03030d6418baceccf148a9dfe57 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Committing config change with OpId 1.327: config changed from index 58 to 327, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 327 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.060180 29870 catalog_manager.cc:5526] T d618a03030d6418baceccf148a9dfe57 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index 58 to 327, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 327 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.061043 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet d618a03030d6418baceccf148a9dfe57 with delete_type TABLET_DATA_TOMBSTONED (TS b26f8d7d861f4663bd853aaec02b192c not found in new config with opid_index 327) from {username='slave'} at 127.0.0.1:47298
I20250112 21:06:25.061205 360 tablet_replica.cc:331] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: stopping tablet replica
I20250112 21:06:25.061270 360 raft_consensus.cc:2238] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:25.061311 360 raft_consensus.cc:2267] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:25.061630 360 ts_tablet_manager.cc:1905] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:25.063136 360 ts_tablet_manager.cc:1918] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.326
I20250112 21:06:25.063184 360 log.cc:1198] T d618a03030d6418baceccf148a9dfe57 P b26f8d7d861f4663bd853aaec02b192c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/d618a03030d6418baceccf148a9dfe57
I20250112 21:06:25.063566 29863 catalog_manager.cc:4872] TS b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): tablet d618a03030d6418baceccf148a9dfe57 (table rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]) successfully deleted
I20250112 21:06:25.118785 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:25.122630 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.125555 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.126673 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:25.131964 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.134020 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.148487 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:25.182422 30020 consensus_queue.cc:237] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 516, Committed index: 516, Last appended: 1.516, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 517 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } }
I20250112 21:06:25.182873 31378 raft_consensus.cc:1270] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 516. Preceding OpId from leader: term: 1 index: 517. (index mismatch)
I20250112 21:06:25.183255 31731 consensus_queue.cc:1035] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 517, Last known committed idx: 516, Time since last communication: 0.000s
I20250112 21:06:25.184013 31770 raft_consensus.cc:2949] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.517: config changed from index 100 to 517, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 517 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:25.184159 31378 raft_consensus.cc:2949] T f2ad09a23ec4424187345c4914d19ac4 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Committing config change with OpId 1.517: config changed from index 100 to 517, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 517 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:25.185542 29871 catalog_manager.cc:5526] T f2ad09a23ec4424187345c4914d19ac4 P cbe373da5cea408ea5c635c7f0164873 reported cstate change: config changed from index 100 to 517, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 517 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } }
I20250112 21:06:25.186604 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet f2ad09a23ec4424187345c4914d19ac4 with delete_type TABLET_DATA_TOMBSTONED (TS b26f8d7d861f4663bd853aaec02b192c not found in new config with opid_index 517) from {username='slave'} at 127.0.0.1:47298
I20250112 21:06:25.186699 360 tablet_replica.cc:331] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: stopping tablet replica
I20250112 21:06:25.186754 360 raft_consensus.cc:2238] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:25.186829 360 raft_consensus.cc:2267] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:25.187341 360 ts_tablet_manager.cc:1905] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:25.188721 360 ts_tablet_manager.cc:1918] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.516
I20250112 21:06:25.188783 360 log.cc:1198] T f2ad09a23ec4424187345c4914d19ac4 P b26f8d7d861f4663bd853aaec02b192c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/f2ad09a23ec4424187345c4914d19ac4
I20250112 21:06:25.189579 29863 catalog_manager.cc:4872] TS b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): tablet f2ad09a23ec4424187345c4914d19ac4 (table rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]) successfully deleted
I20250112 21:06:25.317239 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:25.319792 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.321593 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.324564 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.324730 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:25.342905 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.343655 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:25.390720 30282 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 417, Committed index: 417, Last appended: 1.417, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:25.391311 31509 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 417. Preceding OpId from leader: term: 1 index: 418. (index mismatch)
W20250112 21:06:25.391393 30216 consensus_peers.cc:487] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fdb28e2f5c044d70bc87f9654504832c. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:25.391525 31997 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 418, Last known committed idx: 417, Time since last communication: 0.000s
I20250112 21:06:25.392148 558 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.418: config changed from index 382 to 418, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.392251 31509 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.418: config changed from index 382 to 418, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.393450 29870 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 382 to 418, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.422425 30151 consensus_queue.cc:237] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 348, Committed index: 348, Last appended: 1.348, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:25.422822 30282 raft_consensus.cc:1270] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 348. Preceding OpId from leader: term: 1 index: 349. (index mismatch)
W20250112 21:06:25.423027 30087 consensus_peers.cc:487] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c -> Peer 142e8c9809704200abdf893a5d130c23 (127.26.82.68:42459): Couldn't send request to peer 142e8c9809704200abdf893a5d130c23. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c650f14b4daa4c6682da63dec3e3ad6f. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:25.423054 648 consensus_queue.cc:1035] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 349, Last known committed idx: 348, Time since last communication: 0.000s
I20250112 21:06:25.423854 31759 raft_consensus.cc:2949] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.349: config changed from index -1 to 349, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:25.424091 30282 raft_consensus.cc:2949] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Committing config change with OpId 1.349: config changed from index -1 to 349, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New config: { opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:25.425143 29871 catalog_manager.cc:5526] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index -1 to 349, VOTER 142e8c9809704200abdf893a5d130c23 (127.26.82.68) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:25.462996 30281 consensus_queue.cc:237] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
W20250112 21:06:25.463704 30216 consensus_peers.cc:487] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:25.463986 31376 raft_consensus.cc:1270] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:25.464486 31754 consensus_queue.cc:1035] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:25.465512 31754 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.465668 31376 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.466738 29870 catalog_manager.cc:5526] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 3 to 4, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.502341 30152 consensus_queue.cc:237] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 415, Committed index: 415, Last appended: 1.415, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
W20250112 21:06:25.502920 30085 consensus_peers.cc:487] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c -> Peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): Couldn't send request to peer b26f8d7d861f4663bd853aaec02b192c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58c99277feb2489aa208bb04c7136ba3. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:25.502943 31509 raft_consensus.cc:1270] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 415. Preceding OpId from leader: term: 1 index: 416. (index mismatch)
I20250112 21:06:25.503206 31710 consensus_queue.cc:1035] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 416, Last known committed idx: 415, Time since last communication: 0.000s
I20250112 21:06:25.503975 31710 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.416: config changed from index 404 to 416, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) added. New config: { opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:25.505268 31509 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.416: config changed from index 404 to 416, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) added. New config: { opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:25.505411 29871 catalog_manager.cc:5526] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index 404 to 416, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:25.525488 30152 consensus_queue.cc:237] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 466, Committed index: 466, Last appended: 1.466, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:25.526080 31509 raft_consensus.cc:1270] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 466. Preceding OpId from leader: term: 1 index: 467. (index mismatch)
W20250112 21:06:25.526199 30085 consensus_peers.cc:487] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c -> Peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): Couldn't send request to peer aa93c8a23781463188b9ecba6c3bbf79. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e8ccac51e3343d1b7d180d2fc09101f. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:25.526322 31774 consensus_queue.cc:1035] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 467, Last known committed idx: 466, Time since last communication: 0.000s
I20250112 21:06:25.527117 31710 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.467: config changed from index 319 to 467, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.528286 29871 catalog_manager.cc:5526] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index 319 to 467, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.529122 31509 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.467: config changed from index 319 to 467, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) added. New config: { opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:25.590534 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.591100 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.596891 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:25.597543 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:25.601639 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.602353 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:25.606385 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.670485 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.675822 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:25.677783 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.677875 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.679137 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.684290 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:25.691628 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:25.770212 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.771900 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:25.772466 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.776335 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:25.777200 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.777392 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:25.800060 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.822556 822 ts_tablet_manager.cc:927] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Initiating tablet copy from peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:25.823104 822 tablet_copy_client.cc:323] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:39003
I20250112 21:06:25.823288 30303 tablet_copy_service.cc:140] P b26f8d7d861f4663bd853aaec02b192c: Received BeginTabletCopySession request for tablet fdb28e2f5c044d70bc87f9654504832c from peer aa93c8a23781463188b9ecba6c3bbf79 ({username='slave'} at 127.26.82.71:47507)
I20250112 21:06:25.823350 30303 tablet_copy_service.cc:161] P b26f8d7d861f4663bd853aaec02b192c: Beginning new tablet copy session on tablet fdb28e2f5c044d70bc87f9654504832c from peer aa93c8a23781463188b9ecba6c3bbf79 at {username='slave'} at 127.26.82.71:47507: session id = aa93c8a23781463188b9ecba6c3bbf79-fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:25.823885 30303 tablet_copy_source_session.cc:215] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:25.825392 822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb28e2f5c044d70bc87f9654504832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:25.827348 822 tablet_copy_client.cc:806] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:25.827473 822 tablet_copy_client.cc:670] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:25.831013 822 tablet_copy_client.cc:538] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:25.831720 822 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap starting.
I20250112 21:06:25.884853 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.890146 822 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap replayed 1/1 log segments. Stats: ops{read=427 overwritten=0 applied=427 ignored=0} inserts{seen=1056 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:25.890549 822 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap complete.
I20250112 21:06:25.890803 822 ts_tablet_manager.cc:1397] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Time spent bootstrapping tablet: real 0.059s user 0.026s sys 0.024s
I20250112 21:06:25.891996 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:25.892032 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.892058 822 raft_consensus.cc:357] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:25.892241 822 raft_consensus.cc:738] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Initialized, Role: FOLLOWER
I20250112 21:06:25.892305 822 consensus_queue.cc:260] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 427, Last appended: 1.427, Last appended by leader: 427, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:25.892472 822 ts_tablet_manager.cc:1428] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:25.893564 30303 tablet_copy_service.cc:342] P b26f8d7d861f4663bd853aaec02b192c: Request end of tablet copy session aa93c8a23781463188b9ecba6c3bbf79-fdb28e2f5c044d70bc87f9654504832c received from {username='slave'} at 127.26.82.71:47507
I20250112 21:06:25.893625 30303 tablet_copy_service.cc:434] P b26f8d7d861f4663bd853aaec02b192c: ending tablet copy session aa93c8a23781463188b9ecba6c3bbf79-fdb28e2f5c044d70bc87f9654504832c on tablet fdb28e2f5c044d70bc87f9654504832c with peer aa93c8a23781463188b9ecba6c3bbf79
I20250112 21:06:25.906975 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.907218 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:25.914486 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:25.915882 853 ts_tablet_manager.cc:927] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:25.916265 853 tablet_copy_client.cc:323] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:25.916467 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:25.917757 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet c650f14b4daa4c6682da63dec3e3ad6f from peer 142e8c9809704200abdf893a5d130c23 ({username='slave'} at 127.26.82.68:57461)
I20250112 21:06:25.917820 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet c650f14b4daa4c6682da63dec3e3ad6f from peer 142e8c9809704200abdf893a5d130c23 at {username='slave'} at 127.26.82.68:57461: session id = 142e8c9809704200abdf893a5d130c23-c650f14b4daa4c6682da63dec3e3ad6f
I20250112 21:06:25.918300 30172 tablet_copy_source_session.cc:215] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:25.919608 853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c650f14b4daa4c6682da63dec3e3ad6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:25.921276 853 tablet_copy_client.cc:806] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:25.921425 853 tablet_copy_client.cc:670] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:25.923076 853 tablet_copy_client.cc:538] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:25.923949 853 tablet_bootstrap.cc:492] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: Bootstrap starting.
I20250112 21:06:25.937963 822 ts_tablet_manager.cc:927] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Initiating tablet copy from peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:25.938158 822 tablet_copy_client.cc:323] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:39003
I20250112 21:06:25.938406 30303 tablet_copy_service.cc:140] P b26f8d7d861f4663bd853aaec02b192c: Received BeginTabletCopySession request for tablet ba45544d32d54e369dd5bd0e379c0cb1 from peer aa93c8a23781463188b9ecba6c3bbf79 ({username='slave'} at 127.26.82.71:47507)
I20250112 21:06:25.938464 30303 tablet_copy_service.cc:161] P b26f8d7d861f4663bd853aaec02b192c: Beginning new tablet copy session on tablet ba45544d32d54e369dd5bd0e379c0cb1 from peer aa93c8a23781463188b9ecba6c3bbf79 at {username='slave'} at 127.26.82.71:47507: session id = aa93c8a23781463188b9ecba6c3bbf79-ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:25.938937 30303 tablet_copy_source_session.cc:215] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:25.939182 822 ts_tablet_manager.cc:1905] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:25.941465 822 ts_tablet_manager.cc:1918] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20250112 21:06:25.941627 822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba45544d32d54e369dd5bd0e379c0cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:25.942355 822 tablet_copy_client.cc:806] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:25.942569 822 tablet_copy_client.cc:670] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:25.943326 822 tablet_copy_client.cc:538] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:25.943945 822 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap starting.
I20250112 21:06:25.953364 822 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:25.953594 822 tablet_bootstrap.cc:492] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap complete.
I20250112 21:06:25.953712 822 ts_tablet_manager.cc:1397] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250112 21:06:25.953904 822 raft_consensus.cc:357] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:25.953994 822 raft_consensus.cc:738] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Initialized, Role: FOLLOWER
I20250112 21:06:25.954042 822 consensus_queue.cc:260] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:25.954150 822 ts_tablet_manager.cc:1428] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:25.954325 30303 tablet_copy_service.cc:342] P b26f8d7d861f4663bd853aaec02b192c: Request end of tablet copy session aa93c8a23781463188b9ecba6c3bbf79-ba45544d32d54e369dd5bd0e379c0cb1 received from {username='slave'} at 127.26.82.71:47507
I20250112 21:06:25.954377 30303 tablet_copy_service.cc:434] P b26f8d7d861f4663bd853aaec02b192c: ending tablet copy session aa93c8a23781463188b9ecba6c3bbf79-ba45544d32d54e369dd5bd0e379c0cb1 on tablet ba45544d32d54e369dd5bd0e379c0cb1 with peer aa93c8a23781463188b9ecba6c3bbf79
I20250112 21:06:25.973423 853 tablet_bootstrap.cc:492] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: Bootstrap replayed 1/1 log segments. Stats: ops{read=358 overwritten=0 applied=357 ignored=0} inserts{seen=1042 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250112 21:06:25.973695 853 tablet_bootstrap.cc:492] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: Bootstrap complete.
I20250112 21:06:25.973874 853 ts_tablet_manager.cc:1397] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: Time spent bootstrapping tablet: real 0.050s user 0.019s sys 0.019s
I20250112 21:06:25.974018 853 raft_consensus.cc:357] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:25.974207 853 raft_consensus.cc:738] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142e8c9809704200abdf893a5d130c23, State: Initialized, Role: FOLLOWER
I20250112 21:06:25.974344 853 consensus_queue.cc:260] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 357, Last appended: 1.358, Last appended by leader: 358, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:25.974599 853 ts_tablet_manager.cc:1428] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:25.975246 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session 142e8c9809704200abdf893a5d130c23-c650f14b4daa4c6682da63dec3e3ad6f received from {username='slave'} at 127.26.82.68:57461
I20250112 21:06:25.975324 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session 142e8c9809704200abdf893a5d130c23-c650f14b4daa4c6682da63dec3e3ad6f on tablet c650f14b4daa4c6682da63dec3e3ad6f with peer 142e8c9809704200abdf893a5d130c23
I20250112 21:06:26.002882 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.007652 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:26.009637 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.013332 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:26.015771 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.016160 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:26.023123 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.035182 897 ts_tablet_manager.cc:927] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:26.035377 897 tablet_copy_client.cc:323] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:26.044935 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet 58c99277feb2489aa208bb04c7136ba3 from peer b26f8d7d861f4663bd853aaec02b192c ({username='slave'} at 127.26.82.67:41817)
I20250112 21:06:26.045007 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet 58c99277feb2489aa208bb04c7136ba3 from peer b26f8d7d861f4663bd853aaec02b192c at {username='slave'} at 127.26.82.67:41817: session id = b26f8d7d861f4663bd853aaec02b192c-58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:26.045521 30172 tablet_copy_source_session.cc:215] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:26.046953 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58c99277feb2489aa208bb04c7136ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:26.053666 897 tablet_copy_client.cc:806] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:26.054013 897 tablet_copy_client.cc:670] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:26.057687 897 tablet_copy_client.cc:538] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:26.059005 897 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: Bootstrap starting.
I20250112 21:06:26.077841 822 ts_tablet_manager.cc:927] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:26.078020 822 tablet_copy_client.cc:323] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:26.078285 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet 8e8ccac51e3343d1b7d180d2fc09101f from peer aa93c8a23781463188b9ecba6c3bbf79 ({username='slave'} at 127.26.82.71:39711)
I20250112 21:06:26.078356 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet 8e8ccac51e3343d1b7d180d2fc09101f from peer aa93c8a23781463188b9ecba6c3bbf79 at {username='slave'} at 127.26.82.71:39711: session id = aa93c8a23781463188b9ecba6c3bbf79-8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:26.078953 30172 tablet_copy_source_session.cc:215] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:26.079591 822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8ccac51e3343d1b7d180d2fc09101f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:26.081846 822 tablet_copy_client.cc:806] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:26.081960 822 tablet_copy_client.cc:670] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:26.088164 822 tablet_copy_client.cc:538] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:26.088919 822 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap starting.
I20250112 21:06:26.136230 897 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: Bootstrap replayed 1/1 log segments. Stats: ops{read=479 overwritten=0 applied=479 ignored=0} inserts{seen=1149 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:26.136519 897 tablet_bootstrap.cc:492] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: Bootstrap complete.
I20250112 21:06:26.136798 897 ts_tablet_manager.cc:1397] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: Time spent bootstrapping tablet: real 0.078s user 0.026s sys 0.027s
I20250112 21:06:26.136978 897 raft_consensus.cc:357] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:26.137142 897 raft_consensus.cc:738] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Initialized, Role: FOLLOWER
I20250112 21:06:26.137259 897 consensus_queue.cc:260] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 479, Last appended: 1.479, Last appended by leader: 479, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:26.137885 897 ts_tablet_manager.cc:1428] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:26.138382 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session b26f8d7d861f4663bd853aaec02b192c-58c99277feb2489aa208bb04c7136ba3 received from {username='slave'} at 127.26.82.67:41817
I20250112 21:06:26.138440 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session b26f8d7d861f4663bd853aaec02b192c-58c99277feb2489aa208bb04c7136ba3 on tablet 58c99277feb2489aa208bb04c7136ba3 with peer b26f8d7d861f4663bd853aaec02b192c
I20250112 21:06:26.179247 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.181342 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.192131 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.197919 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:26.200057 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:26.202903 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:26.208086 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.220485 822 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap replayed 1/1 log segments. Stats: ops{read=550 overwritten=0 applied=550 ignored=0} inserts{seen=1233 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:26.220752 822 tablet_bootstrap.cc:492] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: Bootstrap complete.
I20250112 21:06:26.220919 822 ts_tablet_manager.cc:1397] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: Time spent bootstrapping tablet: real 0.132s user 0.046s sys 0.015s
I20250112 21:06:26.221081 822 raft_consensus.cc:357] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:26.221170 822 raft_consensus.cc:738] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Initialized, Role: FOLLOWER
I20250112 21:06:26.221247 822 consensus_queue.cc:260] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 550, Last appended: 1.550, Last appended by leader: 550, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:26.221603 822 ts_tablet_manager.cc:1428] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:26.221910 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session aa93c8a23781463188b9ecba6c3bbf79-8e8ccac51e3343d1b7d180d2fc09101f received from {username='slave'} at 127.26.82.71:39711
I20250112 21:06:26.221976 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session aa93c8a23781463188b9ecba6c3bbf79-8e8ccac51e3343d1b7d180d2fc09101f on tablet 8e8ccac51e3343d1b7d180d2fc09101f with peer aa93c8a23781463188b9ecba6c3bbf79
I20250112 21:06:26.335528 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.339540 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.343874 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:26.349323 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.351967 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.356899 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:26.370163 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:26.381726 31247 raft_consensus.cc:1212] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.348->[1.349-1.393] Dedup: 1.358->[1.359-1.393]
I20250112 21:06:26.436022 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:26.442162 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:26.442314 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.443372 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.444728 31640 raft_consensus.cc:1212] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.417->[1.418-1.479] Dedup: 1.427->[1.428-1.479]
I20250112 21:06:26.449119 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.456547 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.457310 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:26.464498 31640 raft_consensus.cc:1212] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:26.550558 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.552153 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:26.552278 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.553067 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.557694 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.558944 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:26.574765 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:26.599642 30281 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c"
dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39074
I20250112 21:06:26.599740 30281 raft_consensus.cc:604] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Received request to transfer leadership
W20250112 21:06:26.601761 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:26.606235 30281 raft_consensus.cc:1212] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.415->[1.416-1.565] Dedup: 1.479->[1.480-1.565]
W20250112 21:06:26.606875 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.613286 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.618750 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.626199 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.633672 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:26.644217 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
W20250112 21:06:26.645895 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:26.649583 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.649644 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.650066 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:26.653708 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.657773 31641 raft_consensus.cc:1212] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.466->[1.467-1.585] Dedup: 1.550->[1.551-1.585]
W20250112 21:06:26.658890 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:26.659890 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.663746 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20250112 21:06:26.668421 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.680845 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.695304 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.709713 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.725126 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.739553 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.758018 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.773524 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.794000 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.813570 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.835023 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.858592 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.881103 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.903553 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:26.929033 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:26.940726 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:26.945315 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.947118 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.948752 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.952728 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:26.954596 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:26.954983 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20250112 21:06:26.955507 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:26.974325 30152 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "58c99277feb2489aa208bb04c7136ba3"
dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58194
I20250112 21:06:26.974423 30152 raft_consensus.cc:604] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Received request to transfer leadership
W20250112 21:06:26.981143 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:27.010568 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:27.026047 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:27.029170 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.029399 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.031523 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.037411 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20250112 21:06:27.041035 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:27.044708 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.057852 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
W20250112 21:06:27.070600 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
W20250112 21:06:27.103175 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:27.129267 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:27.131147 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:27.131191 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.131708 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.133514 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.137271 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:27.141291 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.151597 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20250112 21:06:27.170300 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Service unavailable: leader transfer in progress
I20250112 21:06:27.172905 30152 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c650f14b4daa4c6682da63dec3e3ad6f"
dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58212
I20250112 21:06:27.173005 30152 raft_consensus.cc:604] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Received request to transfer leadership
W20250112 21:06:27.175415 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.180859 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.183996 31754 raft_consensus.cc:988] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: : Instructing follower aa93c8a23781463188b9ecba6c3bbf79 to start an election
I20250112 21:06:27.184064 31754 raft_consensus.cc:1076] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Signalling peer aa93c8a23781463188b9ecba6c3bbf79 to start an election
W20250112 21:06:27.184321 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.184458 31641 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c"
dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79"
from {username='slave'} at 127.26.82.67:48985
I20250112 21:06:27.184543 31641 raft_consensus.cc:491] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:27.184605 31641 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.185221 31641 raft_consensus.cc:513] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.185420 31641 leader_election.cc:290] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [CANDIDATE]: Term 2 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003), b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387)
I20250112 21:06:27.185539 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c" candidate_uuid: "aa93c8a23781463188b9ecba6c3bbf79" candidate_term: 2 candidate_status { last_received { term: 1 index: 508 } } ignore_live_leader: true dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:27.185612 30282 raft_consensus.cc:3049] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:27.185634 30282 raft_consensus.cc:738] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:27.185683 30282 consensus_queue.cc:260] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 508, Committed index: 508, Last appended: 1.508, Last appended by leader: 508, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.185748 30282 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.186444 30282 raft_consensus.cc:2463] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa93c8a23781463188b9ecba6c3bbf79 in term 2.
I20250112 21:06:27.186710 31573 leader_election.cc:304] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa93c8a23781463188b9ecba6c3bbf79, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:27.187013 1235 raft_consensus.cc:2798] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Leader election won for term 2
W20250112 21:06:27.188154 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.188437 1235 raft_consensus.cc:695] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 LEADER]: Becoming Leader. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Running, Role: LEADER
I20250112 21:06:27.188565 1235 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 508, Committed index: 508, Last appended: 1.508, Last appended by leader: 508, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.189529 31509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c" candidate_uuid: "aa93c8a23781463188b9ecba6c3bbf79" candidate_term: 2 candidate_status { last_received { term: 1 index: 508 } } ignore_live_leader: true dest_uuid: "b05fc98779d84f9589fe07913e1ba558"
I20250112 21:06:27.189636 31509 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.190122 29879 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 reported cstate change: term changed from 1 to 2, leader changed from b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) to aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71). New cstate: current_term: 2 leader_uuid: "aa93c8a23781463188b9ecba6c3bbf79" committed_config { opid_index: 418 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:27.190389 31509 raft_consensus.cc:2463] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa93c8a23781463188b9ecba6c3bbf79 in term 2.
W20250112 21:06:27.194305 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.201794 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.202157 30243 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47352: Illegal state: replica b26f8d7d861f4663bd853aaec02b192c is not leader of this config: current role FOLLOWER
W20250112 21:06:27.209348 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.217759 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250112 21:06:27.217931 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.218833 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:27.221032 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.222236 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:27.226352 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.227684 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.230152 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.233394 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
W20250112 21:06:27.241662 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.255934 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.271304 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.279733 31509 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Refusing update from remote peer aa93c8a23781463188b9ecba6c3bbf79: Log matching property violated. Preceding OpId in replica: term: 1 index: 508. Preceding OpId from leader: term: 2 index: 510. (index mismatch)
I20250112 21:06:27.279986 1235 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 509, Last known committed idx: 508, Time since last communication: 0.000s
I20250112 21:06:27.281255 922 mvcc.cc:204] Tried to move back new op lower bound from 7113588683895967744 to 7113588683528347648. Current Snapshot: MvccSnapshot[applied={T|T < 7113588683895967744 or (T in {7113588683895967744})}]
I20250112 21:06:27.281881 30282 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Refusing update from remote peer aa93c8a23781463188b9ecba6c3bbf79: Log matching property violated. Preceding OpId in replica: term: 1 index: 508. Preceding OpId from leader: term: 2 index: 510. (index mismatch)
I20250112 21:06:27.282214 625 mvcc.cc:204] Tried to move back new op lower bound from 7113588683895967744 to 7113588683528347648. Current Snapshot: MvccSnapshot[applied={T|T < 7113588683895967744 or (T in {7113588683895967744})}]
I20250112 21:06:27.282327 1236 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 509, Last known committed idx: 508, Time since last communication: 0.000s
W20250112 21:06:27.286957 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.300546 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.302320 30131 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:27.304314 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.309834 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.319272 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.323029 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:27.323174 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:27.324452 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.325937 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250112 21:06:27.338124 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.350709 648 raft_consensus.cc:988] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: : Instructing follower b26f8d7d861f4663bd853aaec02b192c to start an election
I20250112 21:06:27.350857 942 raft_consensus.cc:1076] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Signalling peer b26f8d7d861f4663bd853aaec02b192c to start an election
I20250112 21:06:27.351912 30281 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "58c99277feb2489aa208bb04c7136ba3"
dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
from {username='slave'} at 127.26.82.66:54369
I20250112 21:06:27.352005 30281 raft_consensus.cc:491] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:27.352037 30281 raft_consensus.cc:3054] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.352862 30281 raft_consensus.cc:513] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:27.352990 30281 leader_election.cc:290] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 2 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009), b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387)
I20250112 21:06:27.353158 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c99277feb2489aa208bb04c7136ba3" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 2 candidate_status { last_received { term: 1 index: 621 } } ignore_live_leader: true dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:27.353224 30152 raft_consensus.cc:3049] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:27.353250 30152 raft_consensus.cc:738] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:27.353286 30152 consensus_queue.cc:260] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 621, Committed index: 621, Last appended: 1.621, Last appended by leader: 621, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:27.353343 30152 raft_consensus.cc:3054] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.353646 31509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c99277feb2489aa208bb04c7136ba3" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 2 candidate_status { last_received { term: 1 index: 621 } } ignore_live_leader: true dest_uuid: "b05fc98779d84f9589fe07913e1ba558"
I20250112 21:06:27.353713 31509 raft_consensus.cc:3054] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.354068 30152 raft_consensus.cc:2463] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 2.
I20250112 21:06:27.354353 31509 raft_consensus.cc:2463] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 2.
I20250112 21:06:27.354527 30218 leader_election.cc:304] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:27.354647 559 raft_consensus.cc:2798] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:27.354689 559 raft_consensus.cc:695] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 2 LEADER]: Becoming Leader. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:27.354724 559 consensus_queue.cc:237] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 621, Committed index: 621, Last appended: 1.621, Last appended by leader: 621, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:27.355270 29879 catalog_manager.cc:5526] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c reported cstate change: term changed from 1 to 2, leader changed from 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) to b26f8d7d861f4663bd853aaec02b192c (127.26.82.67). New cstate: current_term: 2 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 416 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:27.355504 30151 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8e8ccac51e3343d1b7d180d2fc09101f"
dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58232
I20250112 21:06:27.355566 30151 raft_consensus.cc:604] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Received request to transfer leadership
W20250112 21:06:27.355588 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.359445 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.362902 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.367584 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.374115 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.376222 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.380709 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.386104 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.393494 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.395905 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.398061 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.398217 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.398833 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.401152 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250112 21:06:27.403019 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.403366 30131 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:27.404397 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20250112 21:06:27.414387 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.415701 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
W20250112 21:06:27.421993 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.423918 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.434602 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.445062 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.446158 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.458915 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56376: Illegal state: replica 04cea27f437b471d8557d2fe649a7a7c is not leader of this config: current role FOLLOWER
W20250112 21:06:27.459718 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.469105 30152 raft_consensus.cc:1270] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 621. Preceding OpId from leader: term: 2 index: 623. (index mismatch)
I20250112 21:06:27.469148 31509 raft_consensus.cc:1270] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 621. Preceding OpId from leader: term: 2 index: 623. (index mismatch)
I20250112 21:06:27.469347 559 consensus_queue.cc:1035] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 622, Last known committed idx: 621, Time since last communication: 0.000s
W20250112 21:06:27.469571 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.469624 559 consensus_queue.cc:1035] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 622, Last known committed idx: 621, Time since last communication: 0.000s
W20250112 21:06:27.473421 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.475430 30131 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:27.475605 31227 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.478295 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:27.482748 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:27.488724 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.489964 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.491734 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250112 21:06:27.493220 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.504411 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.505483 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.518818 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.520789 30282 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ba45544d32d54e369dd5bd0e379c0cb1"
dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39162
I20250112 21:06:27.520876 30282 raft_consensus.cc:604] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Received request to transfer leadership
W20250112 21:06:27.523896 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.543824 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.546437 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
W20250112 21:06:27.565374 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.574220 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Service unavailable: leader transfer in progress
I20250112 21:06:27.584142 30131 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
W20250112 21:06:27.585266 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.585734 942 raft_consensus.cc:988] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: : Instructing follower 142e8c9809704200abdf893a5d130c23 to start an election
I20250112 21:06:27.585784 942 raft_consensus.cc:1076] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Signalling peer 142e8c9809704200abdf893a5d130c23 to start an election
I20250112 21:06:27.585930 31247 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c650f14b4daa4c6682da63dec3e3ad6f"
dest_uuid: "142e8c9809704200abdf893a5d130c23"
from {username='slave'} at 127.26.82.66:33985
I20250112 21:06:27.586038 31247 raft_consensus.cc:491] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:27.586102 31247 raft_consensus.cc:3054] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.586937 31247 raft_consensus.cc:513] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:27.587157 31247 leader_election.cc:290] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [CANDIDATE]: Term 2 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009), b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:27.587309 30152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c650f14b4daa4c6682da63dec3e3ad6f" candidate_uuid: "142e8c9809704200abdf893a5d130c23" candidate_term: 2 candidate_status { last_received { term: 1 index: 520 } } ignore_live_leader: true dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:27.587327 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c650f14b4daa4c6682da63dec3e3ad6f" candidate_uuid: "142e8c9809704200abdf893a5d130c23" candidate_term: 2 candidate_status { last_received { term: 1 index: 520 } } ignore_live_leader: true dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:27.587371 30152 raft_consensus.cc:3049] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:27.587383 30282 raft_consensus.cc:3054] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.587396 30152 raft_consensus.cc:738] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:27.587441 30152 consensus_queue.cc:260] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 520, Committed index: 520, Last appended: 1.520, Last appended by leader: 520, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:27.587520 30152 raft_consensus.cc:3054] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.588157 30282 raft_consensus.cc:2463] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142e8c9809704200abdf893a5d130c23 in term 2.
I20250112 21:06:27.588197 30152 raft_consensus.cc:2463] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142e8c9809704200abdf893a5d130c23 in term 2.
I20250112 21:06:27.588325 31180 leader_election.cc:304] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142e8c9809704200abdf893a5d130c23, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:27.588503 1424 raft_consensus.cc:2798] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:27.589596 1424 raft_consensus.cc:695] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 2 LEADER]: Becoming Leader. State: Replica: 142e8c9809704200abdf893a5d130c23, State: Running, Role: LEADER
I20250112 21:06:27.589705 1424 consensus_queue.cc:237] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 520, Committed index: 520, Last appended: 1.520, Last appended by leader: 520, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:27.592470 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.592788 29870 catalog_manager.cc:5526] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 reported cstate change: term changed from 1 to 2, leader changed from 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) to 142e8c9809704200abdf893a5d130c23 (127.26.82.68). New cstate: current_term: 2 leader_uuid: "142e8c9809704200abdf893a5d130c23" committed_config { opid_index: 349 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:27.593405 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:27.594780 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:27.601385 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:27.603037 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.605206 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56396: Illegal state: replica 04cea27f437b471d8557d2fe649a7a7c is not leader of this config: current role FOLLOWER
I20250112 21:06:27.605276 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250112 21:06:27.609609 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.634631 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
W20250112 21:06:27.637727 30242 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39178: Illegal state: replica b26f8d7d861f4663bd853aaec02b192c is not leader of this config: current role FOLLOWER
W20250112 21:06:27.659108 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.673473 30282 raft_consensus.cc:1270] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Refusing update from remote peer 142e8c9809704200abdf893a5d130c23: Log matching property violated. Preceding OpId in replica: term: 1 index: 520. Preceding OpId from leader: term: 2 index: 522. (index mismatch)
I20250112 21:06:27.673691 1424 consensus_queue.cc:1035] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 521, Last known committed idx: 520, Time since last communication: 0.000s
I20250112 21:06:27.673813 30151 raft_consensus.cc:1270] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Refusing update from remote peer 142e8c9809704200abdf893a5d130c23: Log matching property violated. Preceding OpId in replica: term: 1 index: 520. Preceding OpId from leader: term: 2 index: 522. (index mismatch)
I20250112 21:06:27.674047 1424 consensus_queue.cc:1035] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 521, Last known committed idx: 520, Time since last communication: 0.000s
I20250112 21:06:27.675760 624 mvcc.cc:204] Tried to move back new op lower bound from 7113588685508526080 to 7113588685168959488. Current Snapshot: MvccSnapshot[applied={T|T < 7113588685508526080}]
I20250112 21:06:27.678011 964 mvcc.cc:204] Tried to move back new op lower bound from 7113588685508526080 to 7113588685168959488. Current Snapshot: MvccSnapshot[applied={T|T < 7113588685508526080 or (T in {7113588685508526080})}]
W20250112 21:06:27.686888 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.688716 30131 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:27.692906 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:27.693471 31358 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:27.698227 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:27.703743 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:27.705340 30261 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20250112 21:06:27.714289 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.715953 31489 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:27.743789 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.767683 1228 raft_consensus.cc:988] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: : Instructing follower 6e69a4b1e57242c2ba8e9e959c313e52 to start an election
I20250112 21:06:27.767753 1228 raft_consensus.cc:1076] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Signalling peer 6e69a4b1e57242c2ba8e9e959c313e52 to start an election
I20250112 21:06:27.768455 31376 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ba45544d32d54e369dd5bd0e379c0cb1"
dest_uuid: "6e69a4b1e57242c2ba8e9e959c313e52"
from {username='slave'} at 127.26.82.67:35571
I20250112 21:06:27.768558 31376 raft_consensus.cc:491] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:27.768625 31376 raft_consensus.cc:3054] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.769536 31376 raft_consensus.cc:513] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.769805 31376 leader_election.cc:290] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [CANDIDATE]: Term 2 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003), aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
W20250112 21:06:27.770416 30111 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.770697 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba45544d32d54e369dd5bd0e379c0cb1" candidate_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:27.770781 30282 raft_consensus.cc:3049] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:27.770812 30282 raft_consensus.cc:738] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:27.770864 30282 consensus_queue.cc:260] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.770934 30282 raft_consensus.cc:3054] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.771662 30282 raft_consensus.cc:2463] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e69a4b1e57242c2ba8e9e959c313e52 in term 2.
I20250112 21:06:27.773921 31311 leader_election.cc:304] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e69a4b1e57242c2ba8e9e959c313e52, b26f8d7d861f4663bd853aaec02b192c; no voters:
I20250112 21:06:27.773993 31640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba45544d32d54e369dd5bd0e379c0cb1" candidate_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79"
I20250112 21:06:27.774077 31640 raft_consensus.cc:3054] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.774078 1489 raft_consensus.cc:2798] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:27.774184 1489 raft_consensus.cc:695] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 2 LEADER]: Becoming Leader. State: Replica: 6e69a4b1e57242c2ba8e9e959c313e52, State: Running, Role: LEADER
I20250112 21:06:27.774268 1489 consensus_queue.cc:237] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.774837 31640 raft_consensus.cc:2463] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e69a4b1e57242c2ba8e9e959c313e52 in term 2.
I20250112 21:06:27.775477 29873 catalog_manager.cc:5526] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 reported cstate change: term changed from 1 to 2, leader changed from b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) to 6e69a4b1e57242c2ba8e9e959c313e52 (127.26.82.69). New cstate: current_term: 2 leader_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:27.782013 31640 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 577, Committed index: 577, Last appended: 2.577, Last appended by leader: 508, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 578 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.782655 31509 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Refusing update from remote peer aa93c8a23781463188b9ecba6c3bbf79: Log matching property violated. Preceding OpId in replica: term: 2 index: 577. Preceding OpId from leader: term: 2 index: 578. (index mismatch)
I20250112 21:06:27.782881 1294 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 578, Last known committed idx: 577, Time since last communication: 0.000s
I20250112 21:06:27.783452 1235 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 LEADER]: Committing config change with OpId 2.578: config changed from index 418 to 578, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 578 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:27.783558 31509 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Committing config change with OpId 2.578: config changed from index 418 to 578, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 578 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:27.784665 29873 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 reported cstate change: config changed from index 418 to 578, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New cstate: current_term: 2 leader_uuid: "aa93c8a23781463188b9ecba6c3bbf79" committed_config { opid_index: 578 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:27.785511 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb28e2f5c044d70bc87f9654504832c with delete_type TABLET_DATA_TOMBSTONED (TS b26f8d7d861f4663bd853aaec02b192c not found in new config with opid_index 578) from {username='slave'} at 127.0.0.1:47298
I20250112 21:06:27.794189 1492 tablet_replica.cc:331] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: stopping tablet replica
I20250112 21:06:27.794515 1492 raft_consensus.cc:2238] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:27.794659 1492 pending_rounds.cc:62] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Trying to abort 1 pending ops.
I20250112 21:06:27.794687 1492 pending_rounds.cc:69] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Aborting op as it isn't in flight: id { term: 2 index: 577 } timestamp: 7113588685951508480 op_type: WRITE_OP write_request { tablet_id: "fdb28e2f5c044d70bc87f9654504832c" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\rX\230\027\317^V\250\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""e0913ad93d5d1a86" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7113588685942665216 authz_token { token_data: "\010\371\337\220\274\006\"3\n\005slave\022*\n 2f95c2a091af4ea08b44fc659478a5ec\020\001\030\001 \001(\001" signature: "<redacted>""\014\'\247L\250]6\216\311\302~\374\240\303_\227\333\216`\276\256$z\033\233\035{/\221v\247\026o\036zB8N\324\275\243W\001v\253\257T\232\200\247b\264\237|\372D=k/KU\251\276+" signing_key_seq_num: 0 } } request_id { client_id: "be4fd46e9f78495598f4166b50cf0a39" seq_no: 1774 first_incomplete_seq_no: 1774 attempt_no: 0 }
I20250112 21:06:27.794884 1492 raft_consensus.cc:2267] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:27.795209 1492 ts_tablet_manager.cc:1905] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:27.797782 1492 ts_tablet_manager.cc:1918] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.577
I20250112 21:06:27.797837 1492 log.cc:1198] T fdb28e2f5c044d70bc87f9654504832c P b26f8d7d861f4663bd853aaec02b192c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:27.798352 29863 catalog_manager.cc:4872] TS b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): tablet fdb28e2f5c044d70bc87f9654504832c (table rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]) successfully deleted
W20250112 21:06:27.801017 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.832221 726 raft_consensus.cc:988] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: : Instructing follower b05fc98779d84f9589fe07913e1ba558 to start an election
I20250112 21:06:27.832289 726 raft_consensus.cc:1076] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Signalling peer b05fc98779d84f9589fe07913e1ba558 to start an election
W20250112 21:06:27.832468 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Service unavailable: leader transfer in progress
I20250112 21:06:27.832835 31509 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8e8ccac51e3343d1b7d180d2fc09101f"
dest_uuid: "b05fc98779d84f9589fe07913e1ba558"
from {username='slave'} at 127.26.82.66:39367
I20250112 21:06:27.833302 31509 raft_consensus.cc:491] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:27.833462 31509 raft_consensus.cc:3054] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.835664 31509 raft_consensus.cc:513] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.836090 30149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8ccac51e3343d1b7d180d2fc09101f" candidate_uuid: "b05fc98779d84f9589fe07913e1ba558" candidate_term: 2 candidate_status { last_received { term: 1 index: 642 } } ignore_live_leader: true dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:27.836164 30149 raft_consensus.cc:3049] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:27.836192 30149 raft_consensus.cc:738] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:27.836254 30149 consensus_queue.cc:260] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 642, Committed index: 642, Last appended: 1.642, Last appended by leader: 642, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.836318 31509 leader_election.cc:290] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [CANDIDATE]: Term 2 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009), aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:27.836357 30149 raft_consensus.cc:3054] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.837222 30149 raft_consensus.cc:2463] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b05fc98779d84f9589fe07913e1ba558 in term 2.
I20250112 21:06:27.837481 31444 leader_election.cc:304] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04cea27f437b471d8557d2fe649a7a7c, b05fc98779d84f9589fe07913e1ba558; no voters:
I20250112 21:06:27.837656 1286 raft_consensus.cc:2798] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:27.837822 1286 raft_consensus.cc:695] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 2 LEADER]: Becoming Leader. State: Replica: b05fc98779d84f9589fe07913e1ba558, State: Running, Role: LEADER
I20250112 21:06:27.837914 1286 consensus_queue.cc:237] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 642, Committed index: 642, Last appended: 1.642, Last appended by leader: 642, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:27.838490 29871 catalog_manager.cc:5526] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 reported cstate change: term changed from 1 to 2, leader changed from 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) to b05fc98779d84f9589fe07913e1ba558 (127.26.82.70). New cstate: current_term: 2 leader_uuid: "b05fc98779d84f9589fe07913e1ba558" committed_config { opid_index: 467 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:27.841200 31640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8ccac51e3343d1b7d180d2fc09101f" candidate_uuid: "b05fc98779d84f9589fe07913e1ba558" candidate_term: 2 candidate_status { last_received { term: 1 index: 642 } } ignore_live_leader: true dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79"
I20250112 21:06:27.841272 31640 raft_consensus.cc:3054] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:27.842025 31640 raft_consensus.cc:2463] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b05fc98779d84f9589fe07913e1ba558 in term 2.
W20250112 21:06:27.862206 30112 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56412: Illegal state: replica 04cea27f437b471d8557d2fe649a7a7c is not leader of this config: current role FOLLOWER
I20250112 21:06:27.939849 31640 raft_consensus.cc:1270] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Refusing update from remote peer b05fc98779d84f9589fe07913e1ba558: Log matching property violated. Preceding OpId in replica: term: 1 index: 642. Preceding OpId from leader: term: 2 index: 644. (index mismatch)
I20250112 21:06:27.940052 1286 consensus_queue.cc:1035] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 643, Last known committed idx: 642, Time since last communication: 0.000s
I20250112 21:06:27.941530 30149 raft_consensus.cc:1270] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Refusing update from remote peer b05fc98779d84f9589fe07913e1ba558: Log matching property violated. Preceding OpId in replica: term: 1 index: 642. Preceding OpId from leader: term: 2 index: 644. (index mismatch)
I20250112 21:06:27.942016 1286 consensus_queue.cc:1035] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 643, Last known committed idx: 642, Time since last communication: 0.000s
I20250112 21:06:28.071378 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:28.073249 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.079625 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.086000 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.091217 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.095615 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:28.096223 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:28.147279 30281 consensus_queue.cc:237] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 726, Committed index: 726, Last appended: 2.726, Last appended by leader: 621, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 727 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } }
I20250112 21:06:28.147675 31507 raft_consensus.cc:1270] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 2 index: 726. Preceding OpId from leader: term: 2 index: 727. (index mismatch)
I20250112 21:06:28.147869 1513 consensus_queue.cc:1035] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 727, Last known committed idx: 726, Time since last communication: 0.000s
I20250112 21:06:28.148365 1228 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c [term 2 LEADER]: Committing config change with OpId 2.727: config changed from index 416 to 727, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 727 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:28.148464 31507 raft_consensus.cc:2949] T 58c99277feb2489aa208bb04c7136ba3 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Committing config change with OpId 2.727: config changed from index 416 to 727, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 727 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:28.149708 29876 catalog_manager.cc:5526] T 58c99277feb2489aa208bb04c7136ba3 P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 416 to 727, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 727 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } }
I20250112 21:06:28.150560 30131 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c99277feb2489aa208bb04c7136ba3 with delete_type TABLET_DATA_TOMBSTONED (TS 04cea27f437b471d8557d2fe649a7a7c not found in new config with opid_index 727) from {username='slave'} at 127.0.0.1:56302
I20250112 21:06:28.153652 1556 tablet_replica.cc:331] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: stopping tablet replica
I20250112 21:06:28.153767 1556 raft_consensus.cc:2238] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:28.153867 1556 raft_consensus.cc:2267] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:28.154196 1556 ts_tablet_manager.cc:1905] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:28.165535 1556 ts_tablet_manager.cc:1918] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.726
I20250112 21:06:28.165732 1556 log.cc:1198] T 58c99277feb2489aa208bb04c7136ba3 P 04cea27f437b471d8557d2fe649a7a7c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:28.166404 29865 catalog_manager.cc:4872] TS 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009): tablet 58c99277feb2489aa208bb04c7136ba3 (table rebalance_test_table_0 [id=3b737c246bf24fc6a9b3d528a4c6edb9]) successfully deleted
I20250112 21:06:28.246506 30281 raft_consensus.cc:1270] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Refusing update from remote peer 6e69a4b1e57242c2ba8e9e959c313e52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 21:06:28.247048 1489 consensus_queue.cc:1035] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:28.259163 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:28.260694 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.262590 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.269318 31641 raft_consensus.cc:1270] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Refusing update from remote peer 6e69a4b1e57242c2ba8e9e959c313e52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 21:06:28.269542 1490 consensus_queue.cc:1035] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:28.273242 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.273434 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.280697 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:28.281847 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:28.348456 31246 consensus_queue.cc:237] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 603, Committed index: 603, Last appended: 2.603, Last appended by leader: 520, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 604 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } }
I20250112 21:06:28.348883 30282 raft_consensus.cc:1270] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Refusing update from remote peer 142e8c9809704200abdf893a5d130c23: Log matching property violated. Preceding OpId in replica: term: 2 index: 603. Preceding OpId from leader: term: 2 index: 604. (index mismatch)
I20250112 21:06:28.349188 1424 consensus_queue.cc:1035] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 604, Last known committed idx: 603, Time since last communication: 0.000s
I20250112 21:06:28.349860 1424 raft_consensus.cc:2949] T c650f14b4daa4c6682da63dec3e3ad6f P 142e8c9809704200abdf893a5d130c23 [term 2 LEADER]: Committing config change with OpId 2.604: config changed from index 349 to 604, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 604 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:28.349985 30282 raft_consensus.cc:2949] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Committing config change with OpId 2.604: config changed from index 349 to 604, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 604 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:28.351202 29876 catalog_manager.cc:5526] T c650f14b4daa4c6682da63dec3e3ad6f P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 349 to 604, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "142e8c9809704200abdf893a5d130c23" committed_config { opid_index: 604 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "142e8c9809704200abdf893a5d130c23" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 42459 } } }
I20250112 21:06:28.352018 30131 tablet_service.cc:1514] Processing DeleteTablet for tablet c650f14b4daa4c6682da63dec3e3ad6f with delete_type TABLET_DATA_TOMBSTONED (TS 04cea27f437b471d8557d2fe649a7a7c not found in new config with opid_index 604) from {username='slave'} at 127.0.0.1:56302
I20250112 21:06:28.352146 1556 tablet_replica.cc:331] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: stopping tablet replica
I20250112 21:06:28.352216 1556 raft_consensus.cc:2238] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:28.352361 1556 raft_consensus.cc:2267] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:28.352815 1556 ts_tablet_manager.cc:1905] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:28.356050 1556 ts_tablet_manager.cc:1918] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.603
I20250112 21:06:28.356120 1556 log.cc:1198] T c650f14b4daa4c6682da63dec3e3ad6f P 04cea27f437b471d8557d2fe649a7a7c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/c650f14b4daa4c6682da63dec3e3ad6f
I20250112 21:06:28.356703 29865 catalog_manager.cc:4872] TS 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009): tablet c650f14b4daa4c6682da63dec3e3ad6f (table rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]) successfully deleted
I20250112 21:06:28.451588 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:28.455843 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.457914 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.458982 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:28.464542 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.471637 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:28.483446 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.528544 31508 consensus_queue.cc:237] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 728, Committed index: 728, Last appended: 2.728, Last appended by leader: 642, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 729 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:28.529208 31639 raft_consensus.cc:1270] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Refusing update from remote peer b05fc98779d84f9589fe07913e1ba558: Log matching property violated. Preceding OpId in replica: term: 2 index: 728. Preceding OpId from leader: term: 2 index: 729. (index mismatch)
I20250112 21:06:28.529544 1496 consensus_queue.cc:1035] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 729, Last known committed idx: 728, Time since last communication: 0.000s
I20250112 21:06:28.530244 1496 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 [term 2 LEADER]: Committing config change with OpId 2.729: config changed from index 467 to 729, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 729 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:28.530596 31639 raft_consensus.cc:2949] T 8e8ccac51e3343d1b7d180d2fc09101f P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Committing config change with OpId 2.729: config changed from index 467 to 729, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 729 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:28.531527 29873 catalog_manager.cc:5526] T 8e8ccac51e3343d1b7d180d2fc09101f P b05fc98779d84f9589fe07913e1ba558 reported cstate change: config changed from index 467 to 729, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "b05fc98779d84f9589fe07913e1ba558" committed_config { opid_index: 729 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:28.532509 30131 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8ccac51e3343d1b7d180d2fc09101f with delete_type TABLET_DATA_TOMBSTONED (TS 04cea27f437b471d8557d2fe649a7a7c not found in new config with opid_index 729) from {username='slave'} at 127.0.0.1:56302
I20250112 21:06:28.532624 1556 tablet_replica.cc:331] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: stopping tablet replica
I20250112 21:06:28.532704 1556 raft_consensus.cc:2238] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:28.532753 1556 raft_consensus.cc:2267] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:28.533118 1556 ts_tablet_manager.cc:1905] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:28.534694 1556 ts_tablet_manager.cc:1918] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.728
I20250112 21:06:28.534744 1556 log.cc:1198] T 8e8ccac51e3343d1b7d180d2fc09101f P 04cea27f437b471d8557d2fe649a7a7c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:28.535176 29865 catalog_manager.cc:4872] TS 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009): tablet 8e8ccac51e3343d1b7d180d2fc09101f (table rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]) successfully deleted
I20250112 21:06:28.613209 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.619511 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:28.621007 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:28.621868 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.628760 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.631074 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.636490 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:28.673233 31376 consensus_queue.cc:237] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } }
I20250112 21:06:28.673812 31639 raft_consensus.cc:1270] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Refusing update from remote peer 6e69a4b1e57242c2ba8e9e959c313e52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:28.674011 1490 consensus_queue.cc:1035] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:28.674647 1490 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:28.676432 29870 catalog_manager.cc:5526] T ba45544d32d54e369dd5bd0e379c0cb1 P 6e69a4b1e57242c2ba8e9e959c313e52 reported cstate change: config changed from index 4 to 6, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New cstate: current_term: 2 leader_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:28.677350 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet ba45544d32d54e369dd5bd0e379c0cb1 with delete_type TABLET_DATA_TOMBSTONED (TS b26f8d7d861f4663bd853aaec02b192c not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47298
I20250112 21:06:28.677507 1722 tablet_replica.cc:331] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: stopping tablet replica
I20250112 21:06:28.677609 1722 raft_consensus.cc:2238] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:28.677713 1722 raft_consensus.cc:2267] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:28.678901 31639 raft_consensus.cc:2949] T ba45544d32d54e369dd5bd0e379c0cb1 P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e69a4b1e57242c2ba8e9e959c313e52" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 41053 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } }
I20250112 21:06:28.680018 1722 ts_tablet_manager.cc:1905] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:28.682469 1722 ts_tablet_manager.cc:1918] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20250112 21:06:28.682555 1722 log.cc:1198] T ba45544d32d54e369dd5bd0e379c0cb1 P b26f8d7d861f4663bd853aaec02b192c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:28.683014 29863 catalog_manager.cc:4872] TS b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): tablet ba45544d32d54e369dd5bd0e379c0cb1 (table TestTable [id=9982a2b4bec0443585ea28fb19554c5e]) successfully deleted
I20250112 21:06:28.807351 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:28.812009 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.816772 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.817736 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:28.822062 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.822983 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:28.835585 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.924108 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:28.927697 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.933482 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.941829 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:28.946642 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.950114 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:28.965199 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:29.242048 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:29.245757 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.247200 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.248975 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.255872 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:29.256925 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:29.293984 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.402446 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.407948 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.410778 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:29.415786 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.416222 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:29.419209 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:29.419771 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:30.137856 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:30.138074 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:30.144075 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:30.144246 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:30.144333 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:30.152207 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:30.154484 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
523c6484938d414ab64225a8ff19c01f | 127.26.82.126:36745 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.26.82.84:43909 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
04cea27f437b471d8557d2fe649a7a7c | 127.26.82.66:42009 | HEALTHY | <none> | 4 | 0
142e8c9809704200abdf893a5d130c23 | 127.26.82.68:42459 | HEALTHY | <none> | 1 | 0
6e69a4b1e57242c2ba8e9e959c313e52 | 127.26.82.69:41053 | HEALTHY | <none> | 1 | 0
aa93c8a23781463188b9ecba6c3bbf79 | 127.26.82.71:45067 | HEALTHY | <none> | 1 | 0
b05fc98779d84f9589fe07913e1ba558 | 127.26.82.70:36387 | HEALTHY | <none> | 1 | 0
b26f8d7d861f4663bd853aaec02b192c | 127.26.82.67:39003 | HEALTHY | <none> | 3 | 0
cbe373da5cea408ea5c635c7f0164873 | 127.26.82.65:34885 | HEALTHY | <none> | 5 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.26.82.65 | experimental | 127.26.82.65:34885
local_ip_for_outbound_sockets | 127.26.82.66 | experimental | 127.26.82.66:42009
local_ip_for_outbound_sockets | 127.26.82.67 | experimental | 127.26.82.67:39003
local_ip_for_outbound_sockets | 127.26.82.68 | experimental | 127.26.82.68:42459
local_ip_for_outbound_sockets | 127.26.82.69 | experimental | 127.26.82.69:41053
local_ip_for_outbound_sockets | 127.26.82.70 | experimental | 127.26.82.70:36387
local_ip_for_outbound_sockets | 127.26.82.71 | experimental | 127.26.82.71:45067
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.26.82.65:34885
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.26.82.66:42009
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.26.82.67:39003
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.26.82.68:42459
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.26.82.69:41053
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.26.82.70:36387
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.26.82.71:45067
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.26.82.84:43909 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 2 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 4
First Quartile | 4
Median | 5
Third Quartile | 5
Maximum | 5
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 32
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250112 21:06:30.161725 26953 log_verifier.cc:126] Checking tablet 284e96b0178c4ffd9842c70fbdd1658f
I20250112 21:06:30.201875 26953 log_verifier.cc:177] Verified matching terms for 749 ops in tablet 284e96b0178c4ffd9842c70fbdd1658f
I20250112 21:06:30.201987 26953 log_verifier.cc:126] Checking tablet 3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:30.240044 26953 log_verifier.cc:177] Verified matching terms for 755 ops in tablet 3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:30.240258 26953 log_verifier.cc:126] Checking tablet 58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:30.281883 26953 log_verifier.cc:177] Verified matching terms for 854 ops in tablet 58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:30.281960 26953 log_verifier.cc:126] Checking tablet 6cb3ae487cbe46d58ceecc898e2ff13f
I20250112 21:06:30.323908 26953 log_verifier.cc:177] Verified matching terms for 838 ops in tablet 6cb3ae487cbe46d58ceecc898e2ff13f
I20250112 21:06:30.323979 26953 log_verifier.cc:126] Checking tablet 879a5ea2638e4c8e8df85215be98a1b6
I20250112 21:06:30.364722 26953 log_verifier.cc:177] Verified matching terms for 815 ops in tablet 879a5ea2638e4c8e8df85215be98a1b6
I20250112 21:06:30.364801 26953 log_verifier.cc:126] Checking tablet 8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:30.404227 26953 log_verifier.cc:177] Verified matching terms for 794 ops in tablet 8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:30.404381 26953 log_verifier.cc:126] Checking tablet a9f3456185e541c6abc00b7ec105a045
I20250112 21:06:30.442382 26953 log_verifier.cc:177] Verified matching terms for 745 ops in tablet a9f3456185e541c6abc00b7ec105a045
I20250112 21:06:30.442454 26953 log_verifier.cc:126] Checking tablet b03a6dec8f774eaa83abfede3f9c84f5
I20250112 21:06:30.484735 26953 log_verifier.cc:177] Verified matching terms for 830 ops in tablet b03a6dec8f774eaa83abfede3f9c84f5
I20250112 21:06:30.484812 26953 log_verifier.cc:126] Checking tablet ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:30.495548 26953 log_verifier.cc:177] Verified matching terms for 6 ops in tablet ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:30.495605 26953 log_verifier.cc:126] Checking tablet c650f14b4daa4c6682da63dec3e3ad6f
I20250112 21:06:30.536128 26953 log_verifier.cc:177] Verified matching terms for 781 ops in tablet c650f14b4daa4c6682da63dec3e3ad6f
I20250112 21:06:30.536206 26953 log_verifier.cc:126] Checking tablet d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:30.589900 26953 log_verifier.cc:177] Verified matching terms for 1166 ops in tablet d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:30.590000 26953 log_verifier.cc:126] Checking tablet d618a03030d6418baceccf148a9dfe57
I20250112 21:06:30.630626 26953 log_verifier.cc:177] Verified matching terms for 813 ops in tablet d618a03030d6418baceccf148a9dfe57
I20250112 21:06:30.630702 26953 log_verifier.cc:126] Checking tablet f2ad09a23ec4424187345c4914d19ac4
I20250112 21:06:30.682874 26953 log_verifier.cc:177] Verified matching terms for 1125 ops in tablet f2ad09a23ec4424187345c4914d19ac4
I20250112 21:06:30.682951 26953 log_verifier.cc:126] Checking tablet f71bda1d7cfd4744825f7286fa592f2c
I20250112 21:06:30.727100 26953 log_verifier.cc:177] Verified matching terms for 929 ops in tablet f71bda1d7cfd4744825f7286fa592f2c
I20250112 21:06:30.727175 26953 log_verifier.cc:126] Checking tablet fd61ff11a25e482182da04059a77ed36
I20250112 21:06:30.779903 26953 log_verifier.cc:177] Verified matching terms for 1147 ops in tablet fd61ff11a25e482182da04059a77ed36
I20250112 21:06:30.780001 26953 log_verifier.cc:126] Checking tablet fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:30.819648 26953 log_verifier.cc:177] Verified matching terms for 805 ops in tablet fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:30.820611 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.72:0
--local_ip_for_outbound_sockets=127.26.82.72
--webserver_interface=127.26.82.72
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--builtin_ntp_servers=127.26.82.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:30.896152 1952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:30.896289 1952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:30.896307 1952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:30.897579 1952 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:30.897621 1952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:30.897643 1952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.72
I20250112 21:06:30.899016 1952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.82.72
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36745
--never_fsync=true
--heap_profile_path=/tmp/kudu.1952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.72
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:30.899189 1952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:30.899395 1952 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250112 21:06:30.904263 1952 server_base.cc:1034] running on GCE node
W20250112 21:06:30.905301 1957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:30.905906 1958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:30.906694 1960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:30.906876 1952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:30.907042 1952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:30.908162 1952 hybrid_clock.cc:648] HybridClock initialized: now 1736715990908153 us; error 25 us; skew 500 ppm
I20250112 21:06:30.909298 1952 webserver.cc:458] Webserver started at http://127.26.82.72:42003/ using document root <none> and password file <none>
I20250112 21:06:30.909477 1952 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:30.909521 1952 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:30.909595 1952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:30.910356 1952 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "e1f3f50a96a94199bb81d01e90660552"
format_stamp: "Formatted at 2025-01-12 21:06:30 on dist-test-slave-929r"
I20250112 21:06:30.910595 1952 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "e1f3f50a96a94199bb81d01e90660552"
format_stamp: "Formatted at 2025-01-12 21:06:30 on dist-test-slave-929r"
I20250112 21:06:30.911633 1952 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250112 21:06:30.912241 1966 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:30.912405 1952 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:30.912482 1952 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e1f3f50a96a94199bb81d01e90660552"
format_stamp: "Formatted at 2025-01-12 21:06:30 on dist-test-slave-929r"
I20250112 21:06:30.912537 1952 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:30.927006 1952 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:30.927264 1952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:30.927358 1952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:30.927531 1952 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:30.927788 1952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:30.927819 1952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:30.927852 1952 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:30.927877 1952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:30.933109 1952 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.72:40109
I20250112 21:06:30.933169 2079 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.72:40109 every 8 connection(s)
I20250112 21:06:30.933396 1952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20250112 21:06:30.934974 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 1952
I20250112 21:06:30.935065 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20250112 21:06:30.938320 2080 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36745
I20250112 21:06:30.938396 2080 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:30.938555 2080 heartbeater.cc:510] Master 127.26.82.126:36745 requested a full tablet report, sending...
I20250112 21:06:30.938895 29873 ts_manager.cc:194] Registered new tserver with Master: e1f3f50a96a94199bb81d01e90660552 (127.26.82.72:40109)
I20250112 21:06:30.939457 29873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.72:48041
I20250112 21:06:31.045323 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.045691 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.046998 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.055310 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.055895 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.059072 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.062606 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.063676 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.099900 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.102427 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.103451 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.104327 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.105854 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.108232 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.112185 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.115844 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.133132 31641 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 805, Committed index: 805, Last appended: 2.805, Last appended by leader: 508, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.133688 31507 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Refusing update from remote peer aa93c8a23781463188b9ecba6c3bbf79: Log matching property violated. Preceding OpId in replica: term: 2 index: 805. Preceding OpId from leader: term: 2 index: 806. (index mismatch)
I20250112 21:06:31.134035 2188 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 806, Last known committed idx: 805, Time since last communication: 0.000s
I20250112 21:06:31.134806 2187 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 LEADER]: Committing config change with OpId 2.806: config changed from index 578 to 806, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.134967 31507 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Committing config change with OpId 2.806: config changed from index 578 to 806, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.136023 29879 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 reported cstate change: config changed from index 578 to 806, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New cstate: current_term: 2 leader_uuid: "aa93c8a23781463188b9ecba6c3bbf79" committed_config { opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
W20250112 21:06:31.136930 31575 consensus_peers.cc:487] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 -> Peer e1f3f50a96a94199bb81d01e90660552 (127.26.82.72:40109): Couldn't send request to peer e1f3f50a96a94199bb81d01e90660552. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fdb28e2f5c044d70bc87f9654504832c. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:31.146824 30149 consensus_queue.cc:237] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 749, Committed index: 749, Last appended: 1.749, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.147406 30282 raft_consensus.cc:1270] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Refusing update from remote peer 04cea27f437b471d8557d2fe649a7a7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 749. Preceding OpId from leader: term: 1 index: 750. (index mismatch)
I20250112 21:06:31.147594 942 consensus_queue.cc:1035] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 750, Last known committed idx: 749, Time since last communication: 0.000s
I20250112 21:06:31.148280 942 raft_consensus.cc:2949] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Committing config change with OpId 1.750: config changed from index -1 to 750, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.148453 30282 raft_consensus.cc:2949] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Committing config change with OpId 1.750: config changed from index -1 to 750, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.149762 29871 catalog_manager.cc:5526] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c reported cstate change: config changed from index -1 to 750, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "04cea27f437b471d8557d2fe649a7a7c" committed_config { opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
W20250112 21:06:31.152397 30087 consensus_peers.cc:487] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c -> Peer e1f3f50a96a94199bb81d01e90660552 (127.26.82.72:40109): Couldn't send request to peer e1f3f50a96a94199bb81d01e90660552. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 284e96b0178c4ffd9842c70fbdd1658f. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:31.160718 30020 consensus_queue.cc:237] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 755, Committed index: 755, Last appended: 1.755, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.161218 31507 raft_consensus.cc:1270] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Refusing update from remote peer cbe373da5cea408ea5c635c7f0164873: Log matching property violated. Preceding OpId in replica: term: 1 index: 755. Preceding OpId from leader: term: 1 index: 756. (index mismatch)
I20250112 21:06:31.161394 32284 consensus_queue.cc:1035] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 756, Last known committed idx: 755, Time since last communication: 0.000s
I20250112 21:06:31.162057 2230 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Committing config change with OpId 1.756: config changed from index 108 to 756, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.162161 31507 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Committing config change with OpId 1.756: config changed from index 108 to 756, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.163224 29871 catalog_manager.cc:5526] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 reported cstate change: config changed from index 108 to 756, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "cbe373da5cea408ea5c635c7f0164873" committed_config { opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
W20250112 21:06:31.164917 29956 consensus_peers.cc:487] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 -> Peer e1f3f50a96a94199bb81d01e90660552 (127.26.82.72:40109): Couldn't send request to peer e1f3f50a96a94199bb81d01e90660552. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ba4a23fe43d4e11a9462e8f5edca3e0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:31.173944 30282 consensus_queue.cc:237] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1166, Committed index: 1166, Last appended: 1.1166, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.174508 31641 raft_consensus.cc:1270] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1166. Preceding OpId from leader: term: 1 index: 1167. (index mismatch)
I20250112 21:06:31.174790 1395 consensus_queue.cc:1035] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1167, Last known committed idx: 1166, Time since last communication: 0.000s
I20250112 21:06:31.176129 2249 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Committing config change with OpId 1.1167: config changed from index 306 to 1167, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.176258 31641 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Committing config change with OpId 1.1167: config changed from index 306 to 1167, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New config: { opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
W20250112 21:06:31.177683 30218 consensus_peers.cc:487] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c -> Peer e1f3f50a96a94199bb81d01e90660552 (127.26.82.72:40109): Couldn't send request to peer e1f3f50a96a94199bb81d01e90660552. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0dde2209dbc4c9a9a874f32baff5c5d. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:31.177623 29879 catalog_manager.cc:5526] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c reported cstate change: config changed from index 306 to 1167, VOTER e1f3f50a96a94199bb81d01e90660552 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:31.205792 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.205547 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.206105 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.209237 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.211719 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.215760 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.220705 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.221040 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.251243 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.251591 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.254168 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.261519 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.261842 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.262207 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.262583 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.265417 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.301499 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.301517 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.305869 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.306151 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.306891 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.307387 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.307447 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.317579 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.347924 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.349133 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.349493 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.352622 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.356272 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.358644 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.359923 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.364621 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.397912 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.398394 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.401983 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.405467 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.405934 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.408885 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.409422 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.409488 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.443023 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.445952 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.446467 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.451321 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.452894 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.454051 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.454290 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.460143 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.495082 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.496135 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.496625 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.501749 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.505460 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.505707 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.506625 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.514492 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.544270 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.545737 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.549348 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.549783 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.550346 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.554880 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.558892 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.561560 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.587976 2537 ts_tablet_manager.cc:927] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: Initiating tablet copy from peer cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885)
I20250112 21:06:31.588176 2537 tablet_copy_client.cc:323] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:34885
I20250112 21:06:31.590837 30041 tablet_copy_service.cc:140] P cbe373da5cea408ea5c635c7f0164873: Received BeginTabletCopySession request for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 from peer e1f3f50a96a94199bb81d01e90660552 ({username='slave'} at 127.26.82.72:55909)
I20250112 21:06:31.590893 30041 tablet_copy_service.cc:161] P cbe373da5cea408ea5c635c7f0164873: Beginning new tablet copy session on tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 from peer e1f3f50a96a94199bb81d01e90660552 at {username='slave'} at 127.26.82.72:55909: session id = e1f3f50a96a94199bb81d01e90660552-3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:31.591279 30041 tablet_copy_source_session.cc:215] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:31.591934 2537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:31.593976 2537 tablet_copy_client.cc:806] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:31.594123 2537 tablet_copy_client.cc:670] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:31.596441 2537 tablet_copy_client.cc:538] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:31.597064 2537 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: Bootstrap starting.
I20250112 21:06:31.605175 2540 ts_tablet_manager.cc:927] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: Initiating tablet copy from peer aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:31.605330 2540 tablet_copy_client.cc:323] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.71:45067
I20250112 21:06:31.606377 2537 log.cc:826] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:31.608146 31661 tablet_copy_service.cc:140] P aa93c8a23781463188b9ecba6c3bbf79: Received BeginTabletCopySession request for tablet fdb28e2f5c044d70bc87f9654504832c from peer e1f3f50a96a94199bb81d01e90660552 ({username='slave'} at 127.26.82.72:39627)
I20250112 21:06:31.608222 31661 tablet_copy_service.cc:161] P aa93c8a23781463188b9ecba6c3bbf79: Beginning new tablet copy session on tablet fdb28e2f5c044d70bc87f9654504832c from peer e1f3f50a96a94199bb81d01e90660552 at {username='slave'} at 127.26.82.72:39627: session id = e1f3f50a96a94199bb81d01e90660552-fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:31.608919 31661 tablet_copy_source_session.cc:215] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:31.609323 2540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb28e2f5c044d70bc87f9654504832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:31.610157 2540 tablet_copy_client.cc:806] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:31.610257 2540 tablet_copy_client.cc:670] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:31.612984 2540 tablet_copy_client.cc:538] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:31.613478 2540 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: Bootstrap starting.
I20250112 21:06:31.650411 2537 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: Bootstrap replayed 1/1 log segments. Stats: ops{read=756 overwritten=0 applied=756 ignored=0} inserts{seen=1426 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:31.650701 2537 tablet_bootstrap.cc:492] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: Bootstrap complete.
I20250112 21:06:31.650879 2537 ts_tablet_manager.cc:1397] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: Time spent bootstrapping tablet: real 0.054s user 0.021s sys 0.033s
I20250112 21:06:31.651710 2537 raft_consensus.cc:357] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.651840 2537 raft_consensus.cc:738] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f3f50a96a94199bb81d01e90660552, State: Initialized, Role: FOLLOWER
I20250112 21:06:31.651937 2537 consensus_queue.cc:260] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 756, Last appended: 1.756, Last appended by leader: 756, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.652192 2537 ts_tablet_manager.cc:1428] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250112 21:06:31.652273 2080 heartbeater.cc:502] Master 127.26.82.126:36745 was elected leader, sending a full tablet report...
I20250112 21:06:31.652578 30041 tablet_copy_service.cc:342] P cbe373da5cea408ea5c635c7f0164873: Request end of tablet copy session e1f3f50a96a94199bb81d01e90660552-3ba4a23fe43d4e11a9462e8f5edca3e0 received from {username='slave'} at 127.26.82.72:55909
I20250112 21:06:31.652634 30041 tablet_copy_service.cc:434] P cbe373da5cea408ea5c635c7f0164873: ending tablet copy session e1f3f50a96a94199bb81d01e90660552-3ba4a23fe43d4e11a9462e8f5edca3e0 on tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 with peer e1f3f50a96a94199bb81d01e90660552
I20250112 21:06:31.665279 2540 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: Bootstrap replayed 1/1 log segments. Stats: ops{read=806 overwritten=0 applied=806 ignored=0} inserts{seen=1432 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:31.665468 2540 tablet_bootstrap.cc:492] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: Bootstrap complete.
I20250112 21:06:31.665616 2540 ts_tablet_manager.cc:1397] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: Time spent bootstrapping tablet: real 0.052s user 0.035s sys 0.016s
I20250112 21:06:31.665735 2540 raft_consensus.cc:357] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.665791 2540 raft_consensus.cc:738] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f3f50a96a94199bb81d01e90660552, State: Initialized, Role: FOLLOWER
I20250112 21:06:31.665838 2540 consensus_queue.cc:260] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 806, Last appended: 2.806, Last appended by leader: 806, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.665930 2540 ts_tablet_manager.cc:1428] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:31.666172 31661 tablet_copy_service.cc:342] P aa93c8a23781463188b9ecba6c3bbf79: Request end of tablet copy session e1f3f50a96a94199bb81d01e90660552-fdb28e2f5c044d70bc87f9654504832c received from {username='slave'} at 127.26.82.72:39627
I20250112 21:06:31.666239 31661 tablet_copy_service.cc:434] P aa93c8a23781463188b9ecba6c3bbf79: ending tablet copy session e1f3f50a96a94199bb81d01e90660552-fdb28e2f5c044d70bc87f9654504832c on tablet fdb28e2f5c044d70bc87f9654504832c with peer e1f3f50a96a94199bb81d01e90660552
I20250112 21:06:31.706079 2540 ts_tablet_manager.cc:927] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: Initiating tablet copy from peer 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009)
I20250112 21:06:31.706202 2540 tablet_copy_client.cc:323] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42009
I20250112 21:06:31.708557 30172 tablet_copy_service.cc:140] P 04cea27f437b471d8557d2fe649a7a7c: Received BeginTabletCopySession request for tablet 284e96b0178c4ffd9842c70fbdd1658f from peer e1f3f50a96a94199bb81d01e90660552 ({username='slave'} at 127.26.82.72:51965)
I20250112 21:06:31.708613 30172 tablet_copy_service.cc:161] P 04cea27f437b471d8557d2fe649a7a7c: Beginning new tablet copy session on tablet 284e96b0178c4ffd9842c70fbdd1658f from peer e1f3f50a96a94199bb81d01e90660552 at {username='slave'} at 127.26.82.72:51965: session id = e1f3f50a96a94199bb81d01e90660552-284e96b0178c4ffd9842c70fbdd1658f
I20250112 21:06:31.709038 30172 tablet_copy_source_session.cc:215] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:31.709334 2540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 284e96b0178c4ffd9842c70fbdd1658f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:31.710199 2540 tablet_copy_client.cc:806] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:31.710304 2540 tablet_copy_client.cc:670] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:31.712496 2540 tablet_copy_client.cc:538] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:31.713058 2540 tablet_bootstrap.cc:492] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: Bootstrap starting.
I20250112 21:06:31.717988 2537 ts_tablet_manager.cc:927] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: Initiating tablet copy from peer b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003)
I20250112 21:06:31.718108 2537 tablet_copy_client.cc:323] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:39003
I20250112 21:06:31.720945 30303 tablet_copy_service.cc:140] P b26f8d7d861f4663bd853aaec02b192c: Received BeginTabletCopySession request for tablet d0dde2209dbc4c9a9a874f32baff5c5d from peer e1f3f50a96a94199bb81d01e90660552 ({username='slave'} at 127.26.82.72:58957)
I20250112 21:06:31.720997 30303 tablet_copy_service.cc:161] P b26f8d7d861f4663bd853aaec02b192c: Beginning new tablet copy session on tablet d0dde2209dbc4c9a9a874f32baff5c5d from peer e1f3f50a96a94199bb81d01e90660552 at {username='slave'} at 127.26.82.72:58957: session id = e1f3f50a96a94199bb81d01e90660552-d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:31.721340 30303 tablet_copy_source_session.cc:215] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:31.721612 2537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0dde2209dbc4c9a9a874f32baff5c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:31.722339 2537 tablet_copy_client.cc:806] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:31.722477 2537 tablet_copy_client.cc:670] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:31.725328 2537 tablet_copy_client.cc:538] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:31.725807 2537 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: Bootstrap starting.
I20250112 21:06:31.763252 2540 tablet_bootstrap.cc:492] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: Bootstrap replayed 1/1 log segments. Stats: ops{read=750 overwritten=0 applied=750 ignored=0} inserts{seen=1381 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:31.763439 2540 tablet_bootstrap.cc:492] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: Bootstrap complete.
I20250112 21:06:31.763586 2540 ts_tablet_manager.cc:1397] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: Time spent bootstrapping tablet: real 0.051s user 0.033s sys 0.015s
I20250112 21:06:31.763690 2540 raft_consensus.cc:357] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.763747 2540 raft_consensus.cc:738] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f3f50a96a94199bb81d01e90660552, State: Initialized, Role: FOLLOWER
I20250112 21:06:31.763796 2540 consensus_queue.cc:260] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 750, Last appended: 1.750, Last appended by leader: 750, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.763929 2540 ts_tablet_manager.cc:1428] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552: Time spent starting tablet: real 0.000s user 0.002s sys 0.001s
I20250112 21:06:31.764170 30172 tablet_copy_service.cc:342] P 04cea27f437b471d8557d2fe649a7a7c: Request end of tablet copy session e1f3f50a96a94199bb81d01e90660552-284e96b0178c4ffd9842c70fbdd1658f received from {username='slave'} at 127.26.82.72:51965
I20250112 21:06:31.764226 30172 tablet_copy_service.cc:434] P 04cea27f437b471d8557d2fe649a7a7c: ending tablet copy session e1f3f50a96a94199bb81d01e90660552-284e96b0178c4ffd9842c70fbdd1658f on tablet 284e96b0178c4ffd9842c70fbdd1658f with peer e1f3f50a96a94199bb81d01e90660552
I20250112 21:06:31.802042 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.802495 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.806442 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.813256 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.813565 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.818948 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.819384 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.824035 2537 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: Bootstrap replayed 1/1 log segments. Stats: ops{read=1167 overwritten=0 applied=1167 ignored=0} inserts{seen=1837 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:31.824235 2537 tablet_bootstrap.cc:492] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: Bootstrap complete.
I20250112 21:06:31.824419 2537 ts_tablet_manager.cc:1397] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: Time spent bootstrapping tablet: real 0.099s user 0.060s sys 0.025s
I20250112 21:06:31.824532 2537 raft_consensus.cc:357] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.824587 2537 raft_consensus.cc:738] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f3f50a96a94199bb81d01e90660552, State: Initialized, Role: FOLLOWER
I20250112 21:06:31.824625 2537 consensus_queue.cc:260] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1167, Last appended: 1.1167, Last appended by leader: 1167, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:31.824734 2537 ts_tablet_manager.cc:1428] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552: Time spent starting tablet: real 0.000s user 0.002s sys 0.002s
I20250112 21:06:31.824950 30303 tablet_copy_service.cc:342] P b26f8d7d861f4663bd853aaec02b192c: Request end of tablet copy session e1f3f50a96a94199bb81d01e90660552-d0dde2209dbc4c9a9a874f32baff5c5d received from {username='slave'} at 127.26.82.72:58957
I20250112 21:06:31.825006 30303 tablet_copy_service.cc:434] P b26f8d7d861f4663bd853aaec02b192c: ending tablet copy session e1f3f50a96a94199bb81d01e90660552-d0dde2209dbc4c9a9a874f32baff5c5d on tablet d0dde2209dbc4c9a9a874f32baff5c5d with peer e1f3f50a96a94199bb81d01e90660552
I20250112 21:06:31.826269 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.856489 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.859829 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.861450 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.862246 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.862644 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.863720 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.872448 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.876154 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.909317 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.912801 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.914124 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:31.914808 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.916822 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.918275 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.919355 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.927273 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.960884 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.961221 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:31.962481 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.964244 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:31.968614 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.969091 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:31.974578 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:31.978307 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.012204 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.013289 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.015671 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.016741 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.019730 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.020983 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.022558 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.026054 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.062528 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.063409 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.069695 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.070295 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.070348 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.072374 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.073606 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.078027 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.112921 2034 raft_consensus.cc:1212] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.755->[1.756-1.756] Dedup: 1.756->[]
I20250112 21:06:32.114856 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.117695 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.117921 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.119522 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.119787 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.123394 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.124281 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.130463 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.163421 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.163447 2034 raft_consensus.cc:1212] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1166->[1.1167-1.1167] Dedup: 1.1167->[]
I20250112 21:06:32.164489 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.164884 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.166312 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.166924 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.168114 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.171427 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.171486 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.229889 2034 raft_consensus.cc:1212] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.805->[2.806-2.806] Dedup: 2.806->[]
I20250112 21:06:32.267181 2034 raft_consensus.cc:1212] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.749->[1.750-1.750] Dedup: 1.750->[]
I20250112 21:06:32.414777 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.415997 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.421094 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.421278 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.424983 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.425827 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.425992 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.432926 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.442236 30020 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3ba4a23fe43d4e11a9462e8f5edca3e0"
dest_uuid: "cbe373da5cea408ea5c635c7f0164873"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49076
I20250112 21:06:32.442324 30020 raft_consensus.cc:604] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:32.467199 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.468277 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.472000 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.475390 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.476819 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.477350 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.481690 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.485020 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.518524 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.521497 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.521569 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.521783 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.522151 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.532591 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.533794 2014 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.534193 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.545899 30282 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d0dde2209dbc4c9a9a874f32baff5c5d"
dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39528
I20250112 21:06:32.545975 30282 raft_consensus.cc:604] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:32.553529 1395 raft_consensus.cc:988] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: : Instructing follower e1f3f50a96a94199bb81d01e90660552 to start an election
I20250112 21:06:32.553592 1395 raft_consensus.cc:1076] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Signalling peer e1f3f50a96a94199bb81d01e90660552 to start an election
I20250112 21:06:32.553791 2034 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d0dde2209dbc4c9a9a874f32baff5c5d"
dest_uuid: "e1f3f50a96a94199bb81d01e90660552"
from {username='slave'} at 127.26.82.67:56083
I20250112 21:06:32.553913 2034 raft_consensus.cc:491] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:32.553984 2034 raft_consensus.cc:3054] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.554611 2034 raft_consensus.cc:513] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.554844 2034 leader_election.cc:290] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [CANDIDATE]: Term 2 election: Requested vote from peers b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003), aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:32.555032 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dde2209dbc4c9a9a874f32baff5c5d" candidate_uuid: "e1f3f50a96a94199bb81d01e90660552" candidate_term: 2 candidate_status { last_received { term: 1 index: 1167 } } ignore_live_leader: true dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
I20250112 21:06:32.555110 30282 raft_consensus.cc:3049] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:32.555127 30282 raft_consensus.cc:738] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:32.555167 30282 consensus_queue.cc:260] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1167, Committed index: 1167, Last appended: 1.1167, Last appended by leader: 1167, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.555238 30282 raft_consensus.cc:3054] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.555752 30282 raft_consensus.cc:2463] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f3f50a96a94199bb81d01e90660552 in term 2.
I20250112 21:06:32.555847 31641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dde2209dbc4c9a9a874f32baff5c5d" candidate_uuid: "e1f3f50a96a94199bb81d01e90660552" candidate_term: 2 candidate_status { last_received { term: 1 index: 1167 } } ignore_live_leader: true dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79"
I20250112 21:06:32.555915 31641 raft_consensus.cc:3054] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.556428 31641 raft_consensus.cc:2463] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f3f50a96a94199bb81d01e90660552 in term 2.
I20250112 21:06:32.556596 1967 leader_election.cc:304] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa93c8a23781463188b9ecba6c3bbf79, e1f3f50a96a94199bb81d01e90660552; no voters:
I20250112 21:06:32.556754 2544 raft_consensus.cc:2798] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:32.556895 2544 raft_consensus.cc:695] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 2 LEADER]: Becoming Leader. State: Replica: e1f3f50a96a94199bb81d01e90660552, State: Running, Role: LEADER
I20250112 21:06:32.556985 2544 consensus_queue.cc:237] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1167, Committed index: 1167, Last appended: 1.1167, Last appended by leader: 1167, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.557672 29872 catalog_manager.cc:5526] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 reported cstate change: term changed from 1 to 2, leader changed from b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) to e1f3f50a96a94199bb81d01e90660552 (127.26.82.72). New cstate: current_term: 2 leader_uuid: "e1f3f50a96a94199bb81d01e90660552" committed_config { opid_index: 1167 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:32.571448 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.572484 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.576265 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.578565 30261 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:32.580096 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.582707 2014 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.586588 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.591215 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.624938 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.628559 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.629045 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.629426 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.633458 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.633734 2014 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.640046 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.640566 30261 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:32.651386 31641 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c"
dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37352
I20250112 21:06:32.651468 31641 raft_consensus.cc:604] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 LEADER]: Received request to transfer leadership
I20250112 21:06:32.674172 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.677524 31621 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.678763 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.679225 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.684289 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.684593 2014 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.686877 30001 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:32.686986 30261 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:32.725595 2230 raft_consensus.cc:988] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: : Instructing follower e1f3f50a96a94199bb81d01e90660552 to start an election
I20250112 21:06:32.725672 2230 raft_consensus.cc:1076] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Signalling peer e1f3f50a96a94199bb81d01e90660552 to start an election
I20250112 21:06:32.726027 2034 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ba4a23fe43d4e11a9462e8f5edca3e0"
dest_uuid: "e1f3f50a96a94199bb81d01e90660552"
from {username='slave'} at 127.26.82.65:51387
I20250112 21:06:32.726101 2034 raft_consensus.cc:491] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:32.726127 2034 raft_consensus.cc:3054] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.726449 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.726890 2034 raft_consensus.cc:513] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.727063 2034 leader_election.cc:290] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [CANDIDATE]: Term 2 election: Requested vote from peers cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885), b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387)
I20250112 21:06:32.727716 30020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba4a23fe43d4e11a9462e8f5edca3e0" candidate_uuid: "e1f3f50a96a94199bb81d01e90660552" candidate_term: 2 candidate_status { last_received { term: 1 index: 756 } } ignore_live_leader: true dest_uuid: "cbe373da5cea408ea5c635c7f0164873"
I20250112 21:06:32.727798 30020 raft_consensus.cc:3049] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:32.727826 30020 raft_consensus.cc:738] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cbe373da5cea408ea5c635c7f0164873, State: Running, Role: LEADER
I20250112 21:06:32.727870 30020 consensus_queue.cc:260] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 756, Committed index: 756, Last appended: 1.756, Last appended by leader: 756, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.727928 30020 raft_consensus.cc:3054] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.728317 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.730159 30020 raft_consensus.cc:2463] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f3f50a96a94199bb81d01e90660552 in term 2.
I20250112 21:06:32.730357 1970 leader_election.cc:304] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbe373da5cea408ea5c635c7f0164873, e1f3f50a96a94199bb81d01e90660552; no voters:
I20250112 21:06:32.730437 2544 raft_consensus.cc:2798] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:32.730473 2544 raft_consensus.cc:695] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 2 LEADER]: Becoming Leader. State: Replica: e1f3f50a96a94199bb81d01e90660552, State: Running, Role: LEADER
I20250112 21:06:32.730518 2544 consensus_queue.cc:237] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 756, Committed index: 756, Last appended: 1.756, Last appended by leader: 756, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.731173 29876 catalog_manager.cc:5526] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 reported cstate change: term changed from 1 to 2, leader changed from cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) to e1f3f50a96a94199bb81d01e90660552 (127.26.82.72). New cstate: current_term: 2 leader_uuid: "e1f3f50a96a94199bb81d01e90660552" committed_config { opid_index: 756 OBSOLETE_local: false peers { permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 } } peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:32.732297 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.735076 2188 raft_consensus.cc:988] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: : Instructing follower e1f3f50a96a94199bb81d01e90660552 to start an election
I20250112 21:06:32.735141 2188 raft_consensus.cc:1076] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 LEADER]: Signalling peer e1f3f50a96a94199bb81d01e90660552 to start an election
I20250112 21:06:32.735304 2034 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c"
dest_uuid: "e1f3f50a96a94199bb81d01e90660552"
from {username='slave'} at 127.26.82.71:37537
I20250112 21:06:32.735353 2034 raft_consensus.cc:491] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:32.735376 2034 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:32.736088 2034 raft_consensus.cc:513] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.736229 2034 leader_election.cc:290] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [CANDIDATE]: Term 3 election: Requested vote from peers b05fc98779d84f9589fe07913e1ba558 (127.26.82.70:36387), aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067)
I20250112 21:06:32.736246 30261 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:32.736320 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba4a23fe43d4e11a9462e8f5edca3e0" candidate_uuid: "e1f3f50a96a94199bb81d01e90660552" candidate_term: 2 candidate_status { last_received { term: 1 index: 756 } } ignore_live_leader: true dest_uuid: "b05fc98779d84f9589fe07913e1ba558"
I20250112 21:06:32.736406 31507 raft_consensus.cc:3054] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.737143 31507 raft_consensus.cc:2463] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f3f50a96a94199bb81d01e90660552 in term 2.
I20250112 21:06:32.740311 2014 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:32.740398 31641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c" candidate_uuid: "e1f3f50a96a94199bb81d01e90660552" candidate_term: 3 candidate_status { last_received { term: 2 index: 806 } } ignore_live_leader: true dest_uuid: "aa93c8a23781463188b9ecba6c3bbf79"
I20250112 21:06:32.740476 31641 raft_consensus.cc:3049] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 21:06:32.740501 31641 raft_consensus.cc:738] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aa93c8a23781463188b9ecba6c3bbf79, State: Running, Role: LEADER
I20250112 21:06:32.740540 31641 consensus_queue.cc:260] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 806, Committed index: 806, Last appended: 2.806, Last appended by leader: 806, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.740625 31641 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:32.740948 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.741349 31641 raft_consensus.cc:2463] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f3f50a96a94199bb81d01e90660552 in term 3.
I20250112 21:06:32.741447 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.741576 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb28e2f5c044d70bc87f9654504832c" candidate_uuid: "e1f3f50a96a94199bb81d01e90660552" candidate_term: 3 candidate_status { last_received { term: 2 index: 806 } } ignore_live_leader: true dest_uuid: "b05fc98779d84f9589fe07913e1ba558"
I20250112 21:06:32.741636 31507 raft_consensus.cc:3054] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:32.742228 31507 raft_consensus.cc:2463] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f3f50a96a94199bb81d01e90660552 in term 3.
I20250112 21:06:32.742373 1967 leader_election.cc:304] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b05fc98779d84f9589fe07913e1ba558, e1f3f50a96a94199bb81d01e90660552; no voters:
I20250112 21:06:32.742460 2970 raft_consensus.cc:2798] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 21:06:32.742512 2970 raft_consensus.cc:695] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 3 LEADER]: Becoming Leader. State: Replica: e1f3f50a96a94199bb81d01e90660552, State: Running, Role: LEADER
I20250112 21:06:32.742550 2970 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 806, Committed index: 806, Last appended: 2.806, Last appended by leader: 806, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.743083 29876 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 reported cstate change: term changed from 2 to 3, leader changed from aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) to e1f3f50a96a94199bb81d01e90660552 (127.26.82.72). New cstate: current_term: 3 leader_uuid: "e1f3f50a96a94199bb81d01e90660552" committed_config { opid_index: 806 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:32.743239 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.754621 30149 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "284e96b0178c4ffd9842c70fbdd1658f"
dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58638
I20250112 21:06:32.754688 30149 raft_consensus.cc:604] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:32.779289 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.779399 30131 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.784879 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.785619 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:32.785667 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:32.788903 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:32.789541 30261 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:32.789983 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:32.838991 2211 raft_consensus.cc:988] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: : Instructing follower b26f8d7d861f4663bd853aaec02b192c to start an election
I20250112 21:06:32.839064 2211 raft_consensus.cc:1076] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Signalling peer b26f8d7d861f4663bd853aaec02b192c to start an election
I20250112 21:06:32.839241 30282 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "284e96b0178c4ffd9842c70fbdd1658f"
dest_uuid: "b26f8d7d861f4663bd853aaec02b192c"
from {username='slave'} at 127.26.82.66:54369
I20250112 21:06:32.839323 30282 raft_consensus.cc:491] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:32.839345 30282 raft_consensus.cc:3054] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.839944 30282 raft_consensus.cc:513] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.840054 30282 leader_election.cc:290] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 2 election: Requested vote from peers 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009), e1f3f50a96a94199bb81d01e90660552 (127.26.82.72:40109)
I20250112 21:06:32.840220 30149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "284e96b0178c4ffd9842c70fbdd1658f" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 2 candidate_status { last_received { term: 1 index: 750 } } ignore_live_leader: true dest_uuid: "04cea27f437b471d8557d2fe649a7a7c"
I20250112 21:06:32.840287 30149 raft_consensus.cc:3049] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:32.840304 30149 raft_consensus.cc:738] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 04cea27f437b471d8557d2fe649a7a7c, State: Running, Role: LEADER
I20250112 21:06:32.840267 2034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "284e96b0178c4ffd9842c70fbdd1658f" candidate_uuid: "b26f8d7d861f4663bd853aaec02b192c" candidate_term: 2 candidate_status { last_received { term: 1 index: 750 } } ignore_live_leader: true dest_uuid: "e1f3f50a96a94199bb81d01e90660552"
I20250112 21:06:32.840345 30149 consensus_queue.cc:260] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 750, Committed index: 750, Last appended: 1.750, Last appended by leader: 750, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.840378 2034 raft_consensus.cc:3054] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.840413 30149 raft_consensus.cc:3054] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:32.841025 2034 raft_consensus.cc:2463] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 2.
I20250112 21:06:32.841059 30149 raft_consensus.cc:2463] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26f8d7d861f4663bd853aaec02b192c in term 2.
I20250112 21:06:32.841212 30218 leader_election.cc:304] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b26f8d7d861f4663bd853aaec02b192c, e1f3f50a96a94199bb81d01e90660552; no voters:
I20250112 21:06:32.841305 1395 raft_consensus.cc:2798] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:32.841357 1395 raft_consensus.cc:695] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 2 LEADER]: Becoming Leader. State: Replica: b26f8d7d861f4663bd853aaec02b192c, State: Running, Role: LEADER
I20250112 21:06:32.841406 1395 consensus_queue.cc:237] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 750, Committed index: 750, Last appended: 1.750, Last appended by leader: 750, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:32.841902 29879 catalog_manager.cc:5526] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c reported cstate change: term changed from 1 to 2, leader changed from 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) to b26f8d7d861f4663bd853aaec02b192c (127.26.82.67). New cstate: current_term: 2 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 750 OBSOLETE_local: false peers { permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 } } peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:32.948549 30282 raft_consensus.cc:1270] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 1 index: 1167. Preceding OpId from leader: term: 2 index: 1168. (index mismatch)
I20250112 21:06:32.948843 2544 consensus_queue.cc:1035] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1168, Last known committed idx: 1167, Time since last communication: 0.000s
I20250112 21:06:32.950137 31640 raft_consensus.cc:1270] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 1 index: 1167. Preceding OpId from leader: term: 2 index: 1168. (index mismatch)
I20250112 21:06:32.950347 2970 consensus_queue.cc:1035] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1168, Last known committed idx: 1167, Time since last communication: 0.000s
I20250112 21:06:33.029778 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.030925 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.033977 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.035892 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.036819 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.040203 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.043758 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.049149 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.083015 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.084175 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.087410 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.089697 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.092180 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.092283 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.094096 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.097695 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.117069 2034 consensus_queue.cc:237] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1168, Committed index: 1168, Last appended: 2.1168, Last appended by leader: 1167, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 1169 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:33.117450 31640 raft_consensus.cc:1270] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 2 index: 1168. Preceding OpId from leader: term: 2 index: 1169. (index mismatch)
I20250112 21:06:33.117630 2970 consensus_queue.cc:1035] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1169, Last known committed idx: 1168, Time since last communication: 0.000s
I20250112 21:06:33.118197 2544 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 [term 2 LEADER]: Committing config change with OpId 2.1169: config changed from index 1167 to 1169, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 1169 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.118325 31640 raft_consensus.cc:2949] T d0dde2209dbc4c9a9a874f32baff5c5d P aa93c8a23781463188b9ecba6c3bbf79 [term 2 FOLLOWER]: Committing config change with OpId 2.1169: config changed from index 1167 to 1169, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New config: { opid_index: 1169 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.119375 29872 catalog_manager.cc:5526] T d0dde2209dbc4c9a9a874f32baff5c5d P e1f3f50a96a94199bb81d01e90660552 reported cstate change: config changed from index 1167 to 1169, VOTER b26f8d7d861f4663bd853aaec02b192c (127.26.82.67) evicted. New cstate: current_term: 2 leader_uuid: "e1f3f50a96a94199bb81d01e90660552" committed_config { opid_index: 1169 OBSOLETE_local: false peers { permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.120224 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet d0dde2209dbc4c9a9a874f32baff5c5d with delete_type TABLET_DATA_TOMBSTONED (TS b26f8d7d861f4663bd853aaec02b192c not found in new config with opid_index 1169) from {username='slave'} at 127.0.0.1:47298
I20250112 21:06:33.120427 3255 tablet_replica.cc:331] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: stopping tablet replica
I20250112 21:06:33.120537 3255 raft_consensus.cc:2238] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:33.120605 3255 raft_consensus.cc:2267] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:33.121781 3255 ts_tablet_manager.cc:1905] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:33.123172 3255 ts_tablet_manager.cc:1918] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.1168
I20250112 21:06:33.123234 3255 log.cc:1198] T d0dde2209dbc4c9a9a874f32baff5c5d P b26f8d7d861f4663bd853aaec02b192c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:33.123705 29863 catalog_manager.cc:4872] TS b26f8d7d861f4663bd853aaec02b192c (127.26.82.67:39003): tablet d0dde2209dbc4c9a9a874f32baff5c5d (table rebalance_test_table_1 [id=9cc8c6995e864c56a7139da507f748f3]) successfully deleted
I20250112 21:06:33.146109 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.149303 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.150264 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.153436 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.154342 31509 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 3 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 2 index: 806. Preceding OpId from leader: term: 3 index: 807. (index mismatch)
I20250112 21:06:33.154520 2970 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 807, Last known committed idx: 806, Time since last communication: 0.000s
I20250112 21:06:33.155905 31640 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 3 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 2 index: 806. Preceding OpId from leader: term: 3 index: 807. (index mismatch)
I20250112 21:06:33.156097 2544 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa93c8a23781463188b9ecba6c3bbf79" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 807, Last known committed idx: 806, Time since last communication: 0.000s
I20250112 21:06:33.157197 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.157196 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.157526 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.159581 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.180577 30020 raft_consensus.cc:1270] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 2 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 1 index: 756. Preceding OpId from leader: term: 2 index: 757. (index mismatch)
I20250112 21:06:33.180846 2544 consensus_queue.cc:1035] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe373da5cea408ea5c635c7f0164873" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 34885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 757, Last known committed idx: 756, Time since last communication: 0.000s
I20250112 21:06:33.181902 31509 raft_consensus.cc:1270] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 1 index: 756. Preceding OpId from leader: term: 2 index: 757. (index mismatch)
I20250112 21:06:33.182101 2544 consensus_queue.cc:1035] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 757, Last known committed idx: 756, Time since last communication: 0.000s
I20250112 21:06:33.197716 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.197788 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.203075 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.203523 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.205102 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.206282 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.206903 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.213970 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.295709 30149 raft_consensus.cc:1270] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 750. Preceding OpId from leader: term: 2 index: 751. (index mismatch)
I20250112 21:06:33.295941 1395 consensus_queue.cc:1035] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cea27f437b471d8557d2fe649a7a7c" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 751, Last known committed idx: 750, Time since last communication: 0.000s
I20250112 21:06:33.297003 2034 raft_consensus.cc:1270] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 1 index: 750. Preceding OpId from leader: term: 2 index: 751. (index mismatch)
I20250112 21:06:33.297360 1395 consensus_queue.cc:1035] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 751, Last known committed idx: 750, Time since last communication: 0.000s
I20250112 21:06:33.450915 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.451579 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.452854 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.453534 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.456110 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.460846 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.461225 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.461978 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.482731 2034 consensus_queue.cc:237] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 757, Committed index: 757, Last appended: 2.757, Last appended by leader: 756, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 758 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:33.483140 31509 raft_consensus.cc:1270] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 2 index: 757. Preceding OpId from leader: term: 2 index: 758. (index mismatch)
I20250112 21:06:33.483340 2544 consensus_queue.cc:1035] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 758, Last known committed idx: 757, Time since last communication: 0.000s
I20250112 21:06:33.483983 2970 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 [term 2 LEADER]: Committing config change with OpId 2.758: config changed from index 756 to 758, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 758 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.484140 31509 raft_consensus.cc:2949] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P b05fc98779d84f9589fe07913e1ba558 [term 2 FOLLOWER]: Committing config change with OpId 2.758: config changed from index 756 to 758, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New config: { opid_index: 758 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.485052 29870 catalog_manager.cc:5526] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P e1f3f50a96a94199bb81d01e90660552 reported cstate change: config changed from index 756 to 758, VOTER cbe373da5cea408ea5c635c7f0164873 (127.26.82.65) evicted. New cstate: current_term: 2 leader_uuid: "e1f3f50a96a94199bb81d01e90660552" committed_config { opid_index: 758 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.486138 30001 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 with delete_type TABLET_DATA_TOMBSTONED (TS cbe373da5cea408ea5c635c7f0164873 not found in new config with opid_index 758) from {username='slave'} at 127.0.0.1:37368
I20250112 21:06:33.486341 3398 tablet_replica.cc:331] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: stopping tablet replica
I20250112 21:06:33.486447 3398 raft_consensus.cc:2238] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:33.486505 3398 raft_consensus.cc:2267] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:33.487727 3398 ts_tablet_manager.cc:1905] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:33.489147 3398 ts_tablet_manager.cc:1918] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.757
I20250112 21:06:33.489207 3398 log.cc:1198] T 3ba4a23fe43d4e11a9462e8f5edca3e0 P cbe373da5cea408ea5c635c7f0164873: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:33.489655 29866 catalog_manager.cc:4872] TS cbe373da5cea408ea5c635c7f0164873 (127.26.82.65:34885): tablet 3ba4a23fe43d4e11a9462e8f5edca3e0 (table rebalance_test_table_2 [id=b3bb111b43ed4afa9e0e44ccea27c81e]) successfully deleted
I20250112 21:06:33.516376 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.516624 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.519891 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.521761 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.524714 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.526798 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.530802 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.533258 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.554756 2034 consensus_queue.cc:237] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 807, Committed index: 807, Last appended: 3.807, Last appended by leader: 806, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 808 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:33.555191 31509 raft_consensus.cc:1270] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 3 FOLLOWER]: Refusing update from remote peer e1f3f50a96a94199bb81d01e90660552: Log matching property violated. Preceding OpId in replica: term: 3 index: 807. Preceding OpId from leader: term: 3 index: 808. (index mismatch)
I20250112 21:06:33.555410 2544 consensus_queue.cc:1035] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 808, Last known committed idx: 807, Time since last communication: 0.000s
I20250112 21:06:33.556043 2970 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 [term 3 LEADER]: Committing config change with OpId 3.808: config changed from index 806 to 808, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) evicted. New config: { opid_index: 808 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.556143 31509 raft_consensus.cc:2949] T fdb28e2f5c044d70bc87f9654504832c P b05fc98779d84f9589fe07913e1ba558 [term 3 FOLLOWER]: Committing config change with OpId 3.808: config changed from index 806 to 808, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) evicted. New config: { opid_index: 808 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.557291 29870 catalog_manager.cc:5526] T fdb28e2f5c044d70bc87f9654504832c P e1f3f50a96a94199bb81d01e90660552 reported cstate change: config changed from index 806 to 808, VOTER aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71) evicted. New cstate: current_term: 3 leader_uuid: "e1f3f50a96a94199bb81d01e90660552" committed_config { opid_index: 808 OBSOLETE_local: false peers { permanent_uuid: "b05fc98779d84f9589fe07913e1ba558" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 36387 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.558287 31621 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb28e2f5c044d70bc87f9654504832c with delete_type TABLET_DATA_TOMBSTONED (TS aa93c8a23781463188b9ecba6c3bbf79 not found in new config with opid_index 808) from {username='slave'} at 127.0.0.1:36816
I20250112 21:06:33.558480 3453 tablet_replica.cc:331] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: stopping tablet replica
I20250112 21:06:33.558563 3453 raft_consensus.cc:2238] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:33.558615 3453 raft_consensus.cc:2267] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:33.559841 3453 ts_tablet_manager.cc:1905] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:33.561331 3453 ts_tablet_manager.cc:1918] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.807
I20250112 21:06:33.561393 3453 log.cc:1198] T fdb28e2f5c044d70bc87f9654504832c P aa93c8a23781463188b9ecba6c3bbf79: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:33.561787 29863 catalog_manager.cc:4872] TS aa93c8a23781463188b9ecba6c3bbf79 (127.26.82.71:45067): tablet fdb28e2f5c044d70bc87f9654504832c (table rebalance_test_table_3 [id=2f95c2a091af4ea08b44fc659478a5ec]) successfully deleted
I20250112 21:06:33.584621 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.586743 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.588258 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.591727 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.592957 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.593752 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.594982 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.602255 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.618459 30282 consensus_queue.cc:237] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 751, Committed index: 751, Last appended: 2.751, Last appended by leader: 750, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 752 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } }
I20250112 21:06:33.618862 2034 raft_consensus.cc:1270] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Refusing update from remote peer b26f8d7d861f4663bd853aaec02b192c: Log matching property violated. Preceding OpId in replica: term: 2 index: 751. Preceding OpId from leader: term: 2 index: 752. (index mismatch)
I20250112 21:06:33.619103 1395 consensus_queue.cc:1035] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 752, Last known committed idx: 751, Time since last communication: 0.000s
I20250112 21:06:33.619644 2964 raft_consensus.cc:2949] T 284e96b0178c4ffd9842c70fbdd1658f P b26f8d7d861f4663bd853aaec02b192c [term 2 LEADER]: Committing config change with OpId 2.752: config changed from index 750 to 752, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 752 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.619735 2034 raft_consensus.cc:2949] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 [term 2 FOLLOWER]: Committing config change with OpId 2.752: config changed from index 750 to 752, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New config: { opid_index: 752 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.620851 29871 catalog_manager.cc:5526] T 284e96b0178c4ffd9842c70fbdd1658f P e1f3f50a96a94199bb81d01e90660552 reported cstate change: config changed from index 750 to 752, VOTER 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "b26f8d7d861f4663bd853aaec02b192c" committed_config { opid_index: 752 OBSOLETE_local: false peers { permanent_uuid: "b26f8d7d861f4663bd853aaec02b192c" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 39003 } } peers { permanent_uuid: "e1f3f50a96a94199bb81d01e90660552" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 40109 } } }
I20250112 21:06:33.621659 30131 tablet_service.cc:1514] Processing DeleteTablet for tablet 284e96b0178c4ffd9842c70fbdd1658f with delete_type TABLET_DATA_TOMBSTONED (TS 04cea27f437b471d8557d2fe649a7a7c not found in new config with opid_index 752) from {username='slave'} at 127.0.0.1:56302
I20250112 21:06:33.621824 3507 tablet_replica.cc:331] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: stopping tablet replica
I20250112 21:06:33.621891 3507 raft_consensus.cc:2238] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:33.621946 3507 raft_consensus.cc:2267] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:33.623317 3507 ts_tablet_manager.cc:1905] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:33.624933 3507 ts_tablet_manager.cc:1918] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.751
I20250112 21:06:33.624987 3507 log.cc:1198] T 284e96b0178c4ffd9842c70fbdd1658f P 04cea27f437b471d8557d2fe649a7a7c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/284e96b0178c4ffd9842c70fbdd1658f
I20250112 21:06:33.625484 29865 catalog_manager.cc:4872] TS 04cea27f437b471d8557d2fe649a7a7c (127.26.82.66:42009): tablet 284e96b0178c4ffd9842c70fbdd1658f (table rebalance_test_table_4 [id=647ae10537334714afb53ce0d07b82c5]) successfully deleted
I20250112 21:06:33.655472 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.656952 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.657124 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.659909 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.665021 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.665056 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.665872 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.670143 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.703495 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.704950 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.710589 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.711465 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.714603 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.714743 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.715371 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.720852 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:33.761157 30131 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.761260 31621 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:33.761276 31227 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.766578 2014 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.766686 31489 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.770725 31358 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:33.773074 30261 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:33.781466 30001 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
523c6484938d414ab64225a8ff19c01f | 127.26.82.126:36745 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.26.82.84:43909 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
04cea27f437b471d8557d2fe649a7a7c | 127.26.82.66:42009 | HEALTHY | <none> | 3 | 0
142e8c9809704200abdf893a5d130c23 | 127.26.82.68:42459 | HEALTHY | <none> | 1 | 0
6e69a4b1e57242c2ba8e9e959c313e52 | 127.26.82.69:41053 | HEALTHY | <none> | 1 | 0
aa93c8a23781463188b9ecba6c3bbf79 | 127.26.82.71:45067 | HEALTHY | <none> | 0 | 0
b05fc98779d84f9589fe07913e1ba558 | 127.26.82.70:36387 | HEALTHY | <none> | 1 | 0
b26f8d7d861f4663bd853aaec02b192c | 127.26.82.67:39003 | HEALTHY | <none> | 3 | 0
cbe373da5cea408ea5c635c7f0164873 | 127.26.82.65:34885 | HEALTHY | <none> | 4 | 0
e1f3f50a96a94199bb81d01e90660552 | 127.26.82.72:40109 | HEALTHY | <none> | 3 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.26.82.65 | experimental | 127.26.82.65:34885
local_ip_for_outbound_sockets | 127.26.82.66 | experimental | 127.26.82.66:42009
local_ip_for_outbound_sockets | 127.26.82.67 | experimental | 127.26.82.67:39003
local_ip_for_outbound_sockets | 127.26.82.68 | experimental | 127.26.82.68:42459
local_ip_for_outbound_sockets | 127.26.82.69 | experimental | 127.26.82.69:41053
local_ip_for_outbound_sockets | 127.26.82.70 | experimental | 127.26.82.70:36387
local_ip_for_outbound_sockets | 127.26.82.71 | experimental | 127.26.82.71:45067
local_ip_for_outbound_sockets | 127.26.82.72 | experimental | 127.26.82.72:40109
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.26.82.65:34885
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.26.82.66:42009
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.26.82.67:39003
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.26.82.68:42459
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.26.82.69:41053
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.26.82.70:36387
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.26.82.71:45067
server_dump_info_path | /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.RebalanceParamTest.Rebalance_2.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.26.82.72:40109
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.26.82.84:43909 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 2 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 2 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 4
First Quartile | 4
Median | 4
Third Quartile | 4
Maximum | 4
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 32
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250112 21:06:33.788928 26953 log_verifier.cc:126] Checking tablet 284e96b0178c4ffd9842c70fbdd1658f
I20250112 21:06:33.825466 26953 log_verifier.cc:177] Verified matching terms for 752 ops in tablet 284e96b0178c4ffd9842c70fbdd1658f
I20250112 21:06:33.825542 26953 log_verifier.cc:126] Checking tablet 3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:33.860229 26953 log_verifier.cc:177] Verified matching terms for 758 ops in tablet 3ba4a23fe43d4e11a9462e8f5edca3e0
I20250112 21:06:33.860316 26953 log_verifier.cc:126] Checking tablet 58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:33.895588 26953 log_verifier.cc:177] Verified matching terms for 854 ops in tablet 58c99277feb2489aa208bb04c7136ba3
I20250112 21:06:33.895660 26953 log_verifier.cc:126] Checking tablet 6cb3ae487cbe46d58ceecc898e2ff13f
I20250112 21:06:33.932166 26953 log_verifier.cc:177] Verified matching terms for 838 ops in tablet 6cb3ae487cbe46d58ceecc898e2ff13f
I20250112 21:06:33.932231 26953 log_verifier.cc:126] Checking tablet 879a5ea2638e4c8e8df85215be98a1b6
I20250112 21:06:33.966813 26953 log_verifier.cc:177] Verified matching terms for 815 ops in tablet 879a5ea2638e4c8e8df85215be98a1b6
I20250112 21:06:33.966908 26953 log_verifier.cc:126] Checking tablet 8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:34.001714 26953 log_verifier.cc:177] Verified matching terms for 794 ops in tablet 8e8ccac51e3343d1b7d180d2fc09101f
I20250112 21:06:34.001791 26953 log_verifier.cc:126] Checking tablet a9f3456185e541c6abc00b7ec105a045
I20250112 21:06:34.034960 26953 log_verifier.cc:177] Verified matching terms for 745 ops in tablet a9f3456185e541c6abc00b7ec105a045
I20250112 21:06:34.035037 26953 log_verifier.cc:126] Checking tablet b03a6dec8f774eaa83abfede3f9c84f5
I20250112 21:06:34.071430 26953 log_verifier.cc:177] Verified matching terms for 830 ops in tablet b03a6dec8f774eaa83abfede3f9c84f5
I20250112 21:06:34.071524 26953 log_verifier.cc:126] Checking tablet ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:34.077533 26953 log_verifier.cc:177] Verified matching terms for 6 ops in tablet ba45544d32d54e369dd5bd0e379c0cb1
I20250112 21:06:34.077575 26953 log_verifier.cc:126] Checking tablet c650f14b4daa4c6682da63dec3e3ad6f
I20250112 21:06:34.110980 26953 log_verifier.cc:177] Verified matching terms for 781 ops in tablet c650f14b4daa4c6682da63dec3e3ad6f
I20250112 21:06:34.111052 26953 log_verifier.cc:126] Checking tablet d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:34.161304 26953 log_verifier.cc:177] Verified matching terms for 1169 ops in tablet d0dde2209dbc4c9a9a874f32baff5c5d
I20250112 21:06:34.161408 26953 log_verifier.cc:126] Checking tablet d618a03030d6418baceccf148a9dfe57
I20250112 21:06:34.197321 26953 log_verifier.cc:177] Verified matching terms for 813 ops in tablet d618a03030d6418baceccf148a9dfe57
I20250112 21:06:34.197397 26953 log_verifier.cc:126] Checking tablet f2ad09a23ec4424187345c4914d19ac4
I20250112 21:06:34.244086 26953 log_verifier.cc:177] Verified matching terms for 1125 ops in tablet f2ad09a23ec4424187345c4914d19ac4
I20250112 21:06:34.244174 26953 log_verifier.cc:126] Checking tablet f71bda1d7cfd4744825f7286fa592f2c
I20250112 21:06:34.283747 26953 log_verifier.cc:177] Verified matching terms for 929 ops in tablet f71bda1d7cfd4744825f7286fa592f2c
I20250112 21:06:34.283831 26953 log_verifier.cc:126] Checking tablet fd61ff11a25e482182da04059a77ed36
I20250112 21:06:34.332057 26953 log_verifier.cc:177] Verified matching terms for 1147 ops in tablet fd61ff11a25e482182da04059a77ed36
I20250112 21:06:34.332158 26953 log_verifier.cc:126] Checking tablet fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:34.370376 26953 log_verifier.cc:177] Verified matching terms for 808 ops in tablet fdb28e2f5c044d70bc87f9654504832c
I20250112 21:06:34.370970 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 29939
I20250112 21:06:34.377106 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 30070
I20250112 21:06:34.382880 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 30201
I20250112 21:06:34.388124 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 31165
I20250112 21:06:34.393015 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 31296
I20250112 21:06:34.397495 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 31427
I20250112 21:06:34.402129 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 31558
I20250112 21:06:34.406427 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 1952
I20250112 21:06:34.411001 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 29848
2025-01-12T21:06:34Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/2 (20927 ms)
[----------] 1 test from RebalanceParamTest (20927 ms total)
[----------] 1 test from ConcurrentRebalancersTest
[ RUN ] ConcurrentRebalancersTest.TwoConcurrentRebalancers/0
I20250112 21:06:34.462965 26953 test_util.cc:274] Using random seed: -1276314894
I20250112 21:06:34.463460 26953 ts_itest-base.cc:112] Starting cluster with:
I20250112 21:06:34.463490 26953 ts_itest-base.cc:113] --------------
I20250112 21:06:34.463501 26953 ts_itest-base.cc:114] 8 tablet servers
I20250112 21:06:34.463512 26953 ts_itest-base.cc:115] 3 replicas per TS
I20250112 21:06:34.463536 26953 ts_itest-base.cc:116] --------------
2025-01-12T21:06:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-12T21:06:34Z Disabled control of system clock
I20250112 21:06:34.471549 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:36329
--webserver_interface=127.26.82.126
--webserver_port=0
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.82.126:36329
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20250112 21:06:34.544164 3634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:34.544312 3634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:34.544328 3634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:34.545620 3634 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:34.545662 3634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250112 21:06:34.545677 3634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:34.545689 3634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250112 21:06:34.545701 3634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250112 21:06:34.547398 3634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.82.126:36329
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.82.126:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.82.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:34.547586 3634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:34.547767 3634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:34.551844 3639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.552762 3640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.553576 3642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:34.554279 3634 server_base.cc:1034] running on GCE node
I20250112 21:06:34.554454 3634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:34.554668 3634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:34.555794 3634 hybrid_clock.cc:648] HybridClock initialized: now 1736715994555777 us; error 31 us; skew 500 ppm
I20250112 21:06:34.556907 3634 webserver.cc:458] Webserver started at http://127.26.82.126:44149/ using document root <none> and password file <none>
I20250112 21:06:34.557085 3634 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:34.557132 3634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:34.557214 3634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:34.558044 3634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "8959577239d8410abb4e19c5c6c5ad14"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.558315 3634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "8959577239d8410abb4e19c5c6c5ad14"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.559403 3634 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20250112 21:06:34.560012 3648 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.560163 3634 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.560231 3634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
uuid: "8959577239d8410abb4e19c5c6c5ad14"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.560298 3634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:34.572779 3634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:34.573048 3634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:34.573161 3634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:34.576618 3634 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.126:36329
I20250112 21:06:34.576695 3700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.126:36329 every 8 connection(s)
I20250112 21:06:34.576941 3634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250112 21:06:34.577459 3701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:34.579576 3701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14: Bootstrap starting.
I20250112 21:06:34.580111 3701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:34.580346 3701 log.cc:826] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:34.580885 3701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14: No bootstrap required, opened a new log
I20250112 21:06:34.582216 3701 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8959577239d8410abb4e19c5c6c5ad14" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36329 } }
I20250112 21:06:34.582316 3701 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:34.582336 3701 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8959577239d8410abb4e19c5c6c5ad14, State: Initialized, Role: FOLLOWER
I20250112 21:06:34.582450 3701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8959577239d8410abb4e19c5c6c5ad14" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36329 } }
I20250112 21:06:34.582506 3701 raft_consensus.cc:397] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 21:06:34.582530 3701 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 21:06:34.582576 3701 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:34.583042 3701 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8959577239d8410abb4e19c5c6c5ad14" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36329 } }
I20250112 21:06:34.583127 3701 leader_election.cc:304] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8959577239d8410abb4e19c5c6c5ad14; no voters:
I20250112 21:06:34.583261 3701 leader_election.cc:290] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 21:06:34.583320 3704 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:34.583444 3701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [sys.catalog]: configured and running, proceeding with master startup.
I20250112 21:06:34.583467 3704 raft_consensus.cc:695] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [term 1 LEADER]: Becoming Leader. State: Replica: 8959577239d8410abb4e19c5c6c5ad14, State: Running, Role: LEADER
I20250112 21:06:34.583544 3704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8959577239d8410abb4e19c5c6c5ad14" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36329 } }
I20250112 21:06:34.583904 3706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8959577239d8410abb4e19c5c6c5ad14. Latest consensus state: current_term: 1 leader_uuid: "8959577239d8410abb4e19c5c6c5ad14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8959577239d8410abb4e19c5c6c5ad14" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36329 } } }
I20250112 21:06:34.584002 3706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:34.584060 3705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8959577239d8410abb4e19c5c6c5ad14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8959577239d8410abb4e19c5c6c5ad14" member_type: VOTER last_known_addr { host: "127.26.82.126" port: 36329 } } }
I20250112 21:06:34.584125 3705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14 [sys.catalog]: This master's current role is: LEADER
I20250112 21:06:34.584197 3711 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 21:06:34.584584 3711 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 21:06:34.585038 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 3634
I20250112 21:06:34.585165 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250112 21:06:34.586021 3711 catalog_manager.cc:1348] Generated new cluster ID: bf89dd099fa84d3d8bc26c0659a5f78e
I20250112 21:06:34.586076 3711 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 21:06:34.610622 3711 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 21:06:34.611099 3711 catalog_manager.cc:1505] Loading token signing keys...
I20250112 21:06:34.615868 3711 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 8959577239d8410abb4e19c5c6c5ad14: Generated new TSK 0
I20250112 21:06:34.616022 3711 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 21:06:34.622931 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.65:0
--local_ip_for_outbound_sockets=127.26.82.65
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:34.714792 3725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:34.714920 3725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:34.714938 3725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:34.716082 3725 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:34.716120 3725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:34.716138 3725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.65
I20250112 21:06:34.717396 3725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.82.65
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.3725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.65
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:34.717552 3725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:34.717770 3725 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:34.722232 3730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.722911 3733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.723527 3731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:34.724606 3725 server_base.cc:1034] running on GCE node
I20250112 21:06:34.724785 3725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:34.724974 3725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:34.726130 3725 hybrid_clock.cc:648] HybridClock initialized: now 1736715994726103 us; error 39 us; skew 500 ppm
I20250112 21:06:34.727484 3725 webserver.cc:458] Webserver started at http://127.26.82.65:42535/ using document root <none> and password file <none>
I20250112 21:06:34.727701 3725 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:34.727759 3725 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:34.727855 3725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:34.728924 3725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.729311 3725 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.730768 3725 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:34.731446 3739 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.731597 3725 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:34.731673 3725 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.731729 3725 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:34.750077 3725 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:34.750289 3725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:34.750411 3725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:34.750584 3725 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:34.750892 3725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:34.750931 3725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.750967 3725 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:34.750983 3725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.757088 3725 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.65:42903
I20250112 21:06:34.757123 3852 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.65:42903 every 8 connection(s)
I20250112 21:06:34.757445 3725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250112 21:06:34.762814 3853 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:34.762902 3853 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:34.763090 3853 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:34.763582 3665 ts_manager.cc:194] Registered new tserver with Master: bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:34.764235 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.65:37967
I20250112 21:06:34.767660 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 3725
I20250112 21:06:34.767767 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250112 21:06:34.768793 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.66:0
--local_ip_for_outbound_sockets=127.26.82.66
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:34.840131 3856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:34.840283 3856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:34.840301 3856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:34.841496 3856 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:34.841531 3856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:34.841549 3856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.66
I20250112 21:06:34.842729 3856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.82.66
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.3856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.66
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:34.842871 3856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:34.843032 3856 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:34.847097 3861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.848059 3864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.848790 3862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:34.849607 3856 server_base.cc:1034] running on GCE node
I20250112 21:06:34.849731 3856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:34.849905 3856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:34.851027 3856 hybrid_clock.cc:648] HybridClock initialized: now 1736715994851013 us; error 26 us; skew 500 ppm
I20250112 21:06:34.851961 3856 webserver.cc:458] Webserver started at http://127.26.82.66:40881/ using document root <none> and password file <none>
I20250112 21:06:34.852133 3856 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:34.852173 3856 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:34.852257 3856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:34.853034 3856 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "fd8fa04957924e9f83afd61f9adfb455"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.853291 3856 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "fd8fa04957924e9f83afd61f9adfb455"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.854331 3856 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:34.854904 3870 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.855108 3856 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:34.855175 3856 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "fd8fa04957924e9f83afd61f9adfb455"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.855217 3856 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:34.863780 3856 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:34.863924 3856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:34.864095 3856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:34.864328 3856 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:34.864605 3856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:34.864636 3856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.864678 3856 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:34.864708 3856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.869972 3856 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.66:42181
I20250112 21:06:34.870033 3983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.66:42181 every 8 connection(s)
I20250112 21:06:34.870267 3856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250112 21:06:34.872998 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 3856
I20250112 21:06:34.873126 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250112 21:06:34.874328 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.67:0
--local_ip_for_outbound_sockets=127.26.82.67
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:34.875133 3984 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:34.875218 3984 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:34.875406 3984 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:34.875771 3665 ts_manager.cc:194] Registered new tserver with Master: fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:34.876245 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.66:46459
W20250112 21:06:34.945065 3987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:34.945197 3987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:34.945214 3987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:34.946280 3987 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:34.946312 3987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:34.946331 3987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.67
I20250112 21:06:34.947624 3987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.82.67
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.3987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.67
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:34.947774 3987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:34.947957 3987 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:34.952288 3993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.952903 3992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:34.953545 3995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:34.954677 3987 server_base.cc:1034] running on GCE node
I20250112 21:06:34.954820 3987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:34.955003 3987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:34.956128 3987 hybrid_clock.cc:648] HybridClock initialized: now 1736715994956111 us; error 30 us; skew 500 ppm
I20250112 21:06:34.957141 3987 webserver.cc:458] Webserver started at http://127.26.82.67:35243/ using document root <none> and password file <none>
I20250112 21:06:34.957299 3987 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:34.957343 3987 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:34.957428 3987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:34.958209 3987 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.958516 3987 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.959623 3987 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:34.960217 4001 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.960382 3987 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:34.960443 3987 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
format_stamp: "Formatted at 2025-01-12 21:06:34 on dist-test-slave-929r"
I20250112 21:06:34.960495 3987 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:34.973340 3987 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:34.973492 3987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:34.973579 3987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:34.973711 3987 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:34.973948 3987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:34.973975 3987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.974010 3987 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:34.974036 3987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:34.978621 3987 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.67:33119
I20250112 21:06:34.978664 4114 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.67:33119 every 8 connection(s)
I20250112 21:06:34.978905 3987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250112 21:06:34.982770 4115 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:34.982837 4115 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:34.982964 4115 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:34.983258 3665 ts_manager.cc:194] Registered new tserver with Master: 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:34.983552 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.67:37477
I20250112 21:06:34.988755 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 3987
I20250112 21:06:34.988834 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250112 21:06:34.989753 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.68:0
--local_ip_for_outbound_sockets=127.26.82.68
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:35.061108 4118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:35.061242 4118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:35.061266 4118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:35.062340 4118 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:35.062376 4118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:35.062410 4118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.68
I20250112 21:06:35.063812 4118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.82.68
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.4118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.68
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:35.064028 4118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:35.064239 4118 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:35.068627 4123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.069415 4126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.070508 4124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:35.071350 4118 server_base.cc:1034] running on GCE node
I20250112 21:06:35.071478 4118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:35.071635 4118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:35.072713 4118 hybrid_clock.cc:648] HybridClock initialized: now 1736715995072701 us; error 25 us; skew 500 ppm
I20250112 21:06:35.073729 4118 webserver.cc:458] Webserver started at http://127.26.82.68:40335/ using document root <none> and password file <none>
I20250112 21:06:35.073901 4118 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:35.073952 4118 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:35.074046 4118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:35.074751 4118 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "1d1923072e1c45778fa5afdcfda8144d"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.074982 4118 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "1d1923072e1c45778fa5afdcfda8144d"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.075956 4118 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:35.076478 4132 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.076619 4118 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.076699 4118 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "1d1923072e1c45778fa5afdcfda8144d"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.076759 4118 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:35.084450 4118 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:35.084601 4118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:35.084717 4118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:35.084864 4118 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:35.085131 4118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:35.085170 4118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.085204 4118 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:35.085229 4118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.090245 4118 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.68:33569
I20250112 21:06:35.090306 4245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.68:33569 every 8 connection(s)
I20250112 21:06:35.090541 4118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250112 21:06:35.093291 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 4118
I20250112 21:06:35.093386 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250112 21:06:35.094259 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.69:0
--local_ip_for_outbound_sockets=127.26.82.69
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:35.094594 4246 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:35.094669 4246 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:35.094822 4246 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:35.095153 3665 ts_manager.cc:194] Registered new tserver with Master: 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:35.095595 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.68:53329
W20250112 21:06:35.164441 4249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:35.164578 4249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:35.164605 4249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:35.165796 4249 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:35.165835 4249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:35.165870 4249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.69
I20250112 21:06:35.167245 4249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.82.69
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.4249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.69
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:35.167431 4249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:35.167623 4249 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:35.172164 4254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.172842 4257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.173616 4255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:35.174396 4249 server_base.cc:1034] running on GCE node
I20250112 21:06:35.174526 4249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:35.174672 4249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:35.175798 4249 hybrid_clock.cc:648] HybridClock initialized: now 1736715995175774 us; error 39 us; skew 500 ppm
I20250112 21:06:35.176813 4249 webserver.cc:458] Webserver started at http://127.26.82.69:39055/ using document root <none> and password file <none>
I20250112 21:06:35.176959 4249 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:35.176996 4249 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:35.177057 4249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:35.177798 4249 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "6f7f47d877a8407185013dba2959fddd"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.178082 4249 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "6f7f47d877a8407185013dba2959fddd"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.179075 4249 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20250112 21:06:35.179637 4263 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.179812 4249 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.179934 4249 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "6f7f47d877a8407185013dba2959fddd"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.180006 4249 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:35.187283 4249 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:35.187430 4249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:35.187503 4249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:35.187625 4249 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:35.187837 4249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:35.187865 4249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.187891 4249 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:35.187908 4249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.192695 4249 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.69:40871
I20250112 21:06:35.192747 4376 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.69:40871 every 8 connection(s)
I20250112 21:06:35.192968 4249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250112 21:06:35.197011 4377 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:35.197089 4377 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:35.197244 4377 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:35.197547 3665 ts_manager.cc:194] Registered new tserver with Master: 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:35.197875 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.69:37191
I20250112 21:06:35.198311 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 4249
I20250112 21:06:35.198421 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20250112 21:06:35.199405 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.70:0
--local_ip_for_outbound_sockets=127.26.82.70
--webserver_interface=127.26.82.70
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:35.270730 4380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:35.270897 4380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:35.270915 4380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:35.272140 4380 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:35.272176 4380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:35.272197 4380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.70
I20250112 21:06:35.273537 4380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.82.70
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.4380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.70
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:35.273676 4380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:35.273837 4380 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:35.277915 4386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.278702 4388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.279342 4385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:35.280491 4380 server_base.cc:1034] running on GCE node
I20250112 21:06:35.280609 4380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:35.280802 4380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:35.281924 4380 hybrid_clock.cc:648] HybridClock initialized: now 1736715995281909 us; error 26 us; skew 500 ppm
I20250112 21:06:35.282936 4380 webserver.cc:458] Webserver started at http://127.26.82.70:43459/ using document root <none> and password file <none>
I20250112 21:06:35.283104 4380 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:35.283149 4380 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:35.283265 4380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:35.284020 4380 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.284291 4380 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.285398 4380 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250112 21:06:35.286002 4394 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.286139 4380 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:35.286204 4380 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.286258 4380 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:35.305570 4380 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:35.305757 4380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:35.305855 4380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:35.306010 4380 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:35.306272 4380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:35.306303 4380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.306334 4380 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:35.306360 4380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.311326 4380 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.70:43655
I20250112 21:06:35.311383 4507 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.70:43655 every 8 connection(s)
I20250112 21:06:35.311604 4380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20250112 21:06:35.313236 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 4380
I20250112 21:06:35.313333 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20250112 21:06:35.314395 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.71:0
--local_ip_for_outbound_sockets=127.26.82.71
--webserver_interface=127.26.82.71
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:35.316355 4508 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:35.316435 4508 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:35.316589 4508 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:35.316926 3665 ts_manager.cc:194] Registered new tserver with Master: 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:35.317405 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.70:54519
W20250112 21:06:35.387087 4511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:35.387233 4511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:35.387252 4511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:35.388417 4511 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:35.388454 4511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:35.388476 4511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.71
I20250112 21:06:35.389914 4511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.82.71
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.4511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.71
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:35.390079 4511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:35.390247 4511 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:35.394606 4519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.395180 4517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.395888 4516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:35.397010 4511 server_base.cc:1034] running on GCE node
I20250112 21:06:35.397176 4511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:35.397346 4511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:35.398468 4511 hybrid_clock.cc:648] HybridClock initialized: now 1736715995398460 us; error 34 us; skew 500 ppm
I20250112 21:06:35.399528 4511 webserver.cc:458] Webserver started at http://127.26.82.71:39637/ using document root <none> and password file <none>
I20250112 21:06:35.399704 4511 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:35.399752 4511 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:35.399853 4511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:35.400641 4511 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.400954 4511 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.402005 4511 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:35.402544 4525 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.402693 4511 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:35.402753 4511 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.402807 4511 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:35.413081 4511 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:35.413259 4511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:35.413357 4511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:35.413506 4511 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:35.413758 4511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:35.413787 4511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.413817 4511 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:35.413847 4511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.418856 4511 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.71:37889
I20250112 21:06:35.418900 4638 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.71:37889 every 8 connection(s)
I20250112 21:06:35.419157 4511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20250112 21:06:35.423280 4639 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:35.423362 4639 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:35.423501 4639 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:35.423833 3665 ts_manager.cc:194] Registered new tserver with Master: 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:35.424160 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.71:45289
I20250112 21:06:35.428520 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 4511
I20250112 21:06:35.428612 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20250112 21:06:35.429610 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.72:0
--local_ip_for_outbound_sockets=127.26.82.72
--webserver_interface=127.26.82.72
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:35.504086 4642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:35.504249 4642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:35.504267 4642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:35.505518 4642 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:35.505561 4642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:35.505585 4642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.72
I20250112 21:06:35.507004 4642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.82.72
--webserver_port=0
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.4642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.72
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:35.507207 4642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:35.507405 4642 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:35.511808 4648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.512583 4650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:35.513588 4647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:35.514472 4642 server_base.cc:1034] running on GCE node
I20250112 21:06:35.514627 4642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:35.514816 4642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:35.515936 4642 hybrid_clock.cc:648] HybridClock initialized: now 1736715995515921 us; error 32 us; skew 500 ppm
I20250112 21:06:35.516985 4642 webserver.cc:458] Webserver started at http://127.26.82.72:42879/ using document root <none> and password file <none>
I20250112 21:06:35.517160 4642 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:35.517207 4642 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:35.517298 4642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 21:06:35.518103 4642 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "38c8e8afd3f5411490f56ebf57aee109"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.518368 4642 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "38c8e8afd3f5411490f56ebf57aee109"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.519461 4642 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250112 21:06:35.520061 4656 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.520200 4642 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:35.520267 4642 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "38c8e8afd3f5411490f56ebf57aee109"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:35.520323 4642 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:35.528034 4642 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:35.528209 4642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:35.528306 4642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:35.528506 4642 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:35.528807 4642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:35.528839 4642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.528870 4642 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:35.528894 4642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:35.534036 4642 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.72:37429
I20250112 21:06:35.534065 4769 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.72:37429 every 8 connection(s)
I20250112 21:06:35.534312 4642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20250112 21:06:35.538511 4770 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:35.538584 4770 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:35.538744 4770 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:35.539072 3665 ts_manager.cc:194] Registered new tserver with Master: 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:35.539414 3665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.72:51715
I20250112 21:06:35.543499 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 4642
I20250112 21:06:35.543587 26953 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20250112 21:06:35.544543 26953 external_mini_cluster.cc:934] 8 TS(s) registered with all masters
I20250112 21:06:35.550138 26953 external_mini_cluster.cc:934] 8 TS(s) registered with all masters
I20250112 21:06:35.550202 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 4249
I20250112 21:06:35.555240 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 4380
I20250112 21:06:35.559168 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 4511
I20250112 21:06:35.563211 26953 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskam1Qel/build/release/bin/kudu with pid 4642
I20250112 21:06:35.765038 3853 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:35.876865 3984 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:35.984160 4115 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:36.096261 4246 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:39.317833 3665 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:39.326181 4048 tablet_service.cc:1467] Processing CreateTablet for tablet 979cf479af044430904e610adee50359 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.326455 3787 tablet_service.cc:1467] Processing CreateTablet for tablet 3c95203cf74b4b90867a9a3a334dbb21 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.326539 4048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979cf479af044430904e610adee50359. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.326181 4049 tablet_service.cc:1467] Processing CreateTablet for tablet 3c95203cf74b4b90867a9a3a334dbb21 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.326748 3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c95203cf74b4b90867a9a3a334dbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.326664 3918 tablet_service.cc:1467] Processing CreateTablet for tablet 3c95203cf74b4b90867a9a3a334dbb21 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.326906 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c95203cf74b4b90867a9a3a334dbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.327001 3918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c95203cf74b4b90867a9a3a334dbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.328788 3785 tablet_service.cc:1467] Processing CreateTablet for tablet 979cf479af044430904e610adee50359 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.328891 3785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979cf479af044430904e610adee50359. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.326664 3917 tablet_service.cc:1467] Processing CreateTablet for tablet dd9176ff75364d81a5b3bdd0e7e10673 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.329864 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd9176ff75364d81a5b3bdd0e7e10673. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.330080 3786 tablet_service.cc:1467] Processing CreateTablet for tablet dd9176ff75364d81a5b3bdd0e7e10673 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.330166 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd9176ff75364d81a5b3bdd0e7e10673. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.331080 4792 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:39.331480 4180 tablet_service.cc:1467] Processing CreateTablet for tablet dd9176ff75364d81a5b3bdd0e7e10673 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.331650 4792 tablet_bootstrap.cc:654] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.331758 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd9176ff75364d81a5b3bdd0e7e10673. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.331961 4792 log.cc:826] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:39.332286 4790 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:39.332691 4792 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:39.332759 4179 tablet_service.cc:1467] Processing CreateTablet for tablet 979cf479af044430904e610adee50359 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.332805 4790 tablet_bootstrap.cc:654] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.332876 4179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979cf479af044430904e610adee50359. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.332902 4792 ts_tablet_manager.cc:1397] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:39.333070 4790 log.cc:826] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:39.333341 4791 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:39.333904 4791 tablet_bootstrap.cc:654] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.334133 4791 log.cc:826] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:39.334542 4790 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:39.334596 4790 ts_tablet_manager.cc:1397] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:39.334867 4796 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:39.335139 4792 raft_consensus.cc:357] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.335368 4792 raft_consensus.cc:383] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.335458 4792 raft_consensus.cc:738] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.335650 4792 consensus_queue.cc:260] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.335755 4796 tablet_bootstrap.cc:654] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.336007 4792 ts_tablet_manager.cc:1428] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:39.336187 4796 log.cc:826] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:39.336306 4792 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:39.336807 4792 tablet_bootstrap.cc:654] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.336894 4796 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:39.336973 4796 ts_tablet_manager.cc:1397] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:39.337357 4792 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:39.337276 4790 raft_consensus.cc:357] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.337406 4790 raft_consensus.cc:383] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.337414 4792 ts_tablet_manager.cc:1397] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:39.337432 4790 raft_consensus.cc:738] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.337539 4792 raft_consensus.cc:357] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.337594 4792 raft_consensus.cc:383] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.337613 4792 raft_consensus.cc:738] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.337692 4792 consensus_queue.cc:260] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.337775 4791 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:39.337841 4792 ts_tablet_manager.cc:1428] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:39.337968 4790 consensus_queue.cc:260] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.338249 4790 ts_tablet_manager.cc:1428] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20250112 21:06:39.338330 4790 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:39.338770 4790 tablet_bootstrap.cc:654] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.338887 4791 ts_tablet_manager.cc:1397] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250112 21:06:39.340152 4796 raft_consensus.cc:357] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.340289 4796 raft_consensus.cc:383] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.340320 4796 raft_consensus.cc:738] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.340428 4796 consensus_queue.cc:260] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.340652 4796 ts_tablet_manager.cc:1428] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20250112 21:06:39.340764 4796 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:39.341207 4796 tablet_bootstrap.cc:654] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.341514 4791 raft_consensus.cc:357] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.341629 4791 raft_consensus.cc:383] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.341655 4791 raft_consensus.cc:738] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.341717 4791 consensus_queue.cc:260] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.341912 4791 ts_tablet_manager.cc:1428] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:39.341976 4791 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:39.342339 4791 tablet_bootstrap.cc:654] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.342695 4790 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:39.342751 4790 ts_tablet_manager.cc:1397] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20250112 21:06:39.342921 4790 raft_consensus.cc:357] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.342893 4791 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:39.343022 4790 raft_consensus.cc:383] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.343063 4790 raft_consensus.cc:738] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.343057 4791 ts_tablet_manager.cc:1397] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:39.343137 4790 consensus_queue.cc:260] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.343228 4790 ts_tablet_manager.cc:1428] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.343212 4791 raft_consensus.cc:357] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.343266 4791 raft_consensus.cc:383] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.343339 4791 raft_consensus.cc:738] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
W20250112 21:06:39.343339 4247 tablet.cc:2367] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:39.343429 4791 consensus_queue.cc:260] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.343564 4791 ts_tablet_manager.cc:1428] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.343755 4791 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:39.343829 4796 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:39.343886 4796 ts_tablet_manager.cc:1397] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:39.344038 4796 raft_consensus.cc:357] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.344110 4796 raft_consensus.cc:383] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.344135 4796 raft_consensus.cc:738] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.344141 4791 tablet_bootstrap.cc:654] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.344231 4796 consensus_queue.cc:260] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.344341 4796 ts_tablet_manager.cc:1428] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:39.344724 4791 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:39.344770 4791 ts_tablet_manager.cc:1397] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:39.344955 4791 raft_consensus.cc:357] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.345043 4791 raft_consensus.cc:383] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.345073 4791 raft_consensus.cc:738] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.345118 4791 consensus_queue.cc:260] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.345218 4791 ts_tablet_manager.cc:1428] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.353825 4797 raft_consensus.cc:491] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:39.353886 4797 raft_consensus.cc:513] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.354125 4797 leader_election.cc:290] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:39.357390 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:39.357427 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:39.357481 4069 raft_consensus.cc:2463] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:39.357553 3807 raft_consensus.cc:2463] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:39.357975 3871 leader_election.cc:304] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:39.358204 4797 raft_consensus.cc:2798] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:39.358249 4797 raft_consensus.cc:491] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:39.358268 4797 raft_consensus.cc:3054] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.358733 4797 raft_consensus.cc:513] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.358829 4797 leader_election.cc:290] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:39.358965 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:39.359007 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:39.359031 4069 raft_consensus.cc:3054] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.359051 3807 raft_consensus.cc:3054] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.359530 4069 raft_consensus.cc:2463] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:39.359536 3807 raft_consensus.cc:2463] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:39.359709 3871 leader_election.cc:304] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:39.359826 4797 raft_consensus.cc:2798] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:39.359891 4797 raft_consensus.cc:695] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:39.359963 4797 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.360512 3664 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
W20250112 21:06:39.373097 3985 tablet.cc:2367] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 21:06:39.409734 4802 raft_consensus.cc:491] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:39.409821 4802 raft_consensus.cc:513] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.410079 4802 leader_election.cc:290] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:39.412587 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979cf479af044430904e610adee50359" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:39.412674 4069 raft_consensus.cc:2463] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:39.412613 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979cf479af044430904e610adee50359" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:39.412720 3807 raft_consensus.cc:2463] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:39.412879 4133 leader_election.cc:304] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:39.413028 4802 raft_consensus.cc:2798] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:39.413096 4802 raft_consensus.cc:491] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:39.413138 4802 raft_consensus.cc:3054] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.413966 4802 raft_consensus.cc:513] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.414088 4802 leader_election.cc:290] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:39.414227 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979cf479af044430904e610adee50359" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:39.414237 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979cf479af044430904e610adee50359" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:39.414288 3807 raft_consensus.cc:3054] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.414289 4069 raft_consensus.cc:3054] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.414788 3807 raft_consensus.cc:2463] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:39.414830 4069 raft_consensus.cc:2463] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:39.414979 4134 leader_election.cc:304] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:39.415066 4802 raft_consensus.cc:2798] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:39.415196 4802 raft_consensus.cc:695] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:39.415273 4802 consensus_queue.cc:237] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.415853 3664 catalog_manager.cc:5526] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:39.455462 4805 raft_consensus.cc:491] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:39.455528 4805 raft_consensus.cc:513] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.455782 4805 leader_election.cc:290] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:39.458334 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9176ff75364d81a5b3bdd0e7e10673" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:39.458425 3938 raft_consensus.cc:2463] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:39.458400 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9176ff75364d81a5b3bdd0e7e10673" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:39.458485 4200 raft_consensus.cc:2463] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:39.458622 3741 leader_election.cc:304] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:39.458731 4805 raft_consensus.cc:2798] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:39.458782 4805 raft_consensus.cc:491] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:39.458808 4805 raft_consensus.cc:3054] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.459390 4805 raft_consensus.cc:513] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.459540 4805 leader_election.cc:290] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:39.459709 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9176ff75364d81a5b3bdd0e7e10673" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:39.459774 4200 raft_consensus.cc:3054] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.459728 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9176ff75364d81a5b3bdd0e7e10673" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:39.459892 3938 raft_consensus.cc:3054] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.460354 4200 raft_consensus.cc:2463] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:39.460538 3938 raft_consensus.cc:2463] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:39.460533 3740 leader_election.cc:304] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:39.460618 4805 raft_consensus.cc:2798] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:39.460774 4805 raft_consensus.cc:695] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:39.460870 4805 consensus_queue.cc:237] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.461486 3664 catalog_manager.cc:5526] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
W20250112 21:06:39.481623 4116 tablet.cc:2367] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250112 21:06:39.510360 3854 tablet.cc:2367] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250112 21:06:39.545773 4819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:39.545922 4819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:39.547223 4819 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20250112 21:06:39.583760 3807 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.584704 4797 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.585495 4069 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.585764 4797 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.586848 3806 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.587090 4802 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.587306 3938 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.587772 4068 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.587834 4200 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.587855 4817 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.588218 4805 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.589088 4813 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.590095 4858 mvcc.cc:204] Tried to move back new op lower bound from 7113588734307336192 to 7113588733792264192. Current Snapshot: MvccSnapshot[applied={T|T < 7113588734307336192}]
I20250112 21:06:39.592788 4852 mvcc.cc:204] Tried to move back new op lower bound from 7113588734328242176 to 7113588733605408768. Current Snapshot: MvccSnapshot[applied={T|T < 7113588734305239040}]
I20250112 21:06:39.595635 4864 mvcc.cc:204] Tried to move back new op lower bound from 7113588734305239040 to 7113588733605408768. Current Snapshot: MvccSnapshot[applied={T|T < 7113588734305239040}]
I20250112 21:06:39.596614 4851 mvcc.cc:204] Tried to move back new op lower bound from 7113588734292639744 to 7113588733378531328. Current Snapshot: MvccSnapshot[applied={T|T < 7113588734292639744}]
INSERT report
rows total: 2000
time total: 28.7347 ms
time per row: 0.0143673 ms
I20250112 21:06:39.604677 3664 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:39.605952 4180 tablet_service.cc:1467] Processing CreateTablet for tablet 87835ddc19954987b0f17f6aa0370c10 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.605971 3918 tablet_service.cc:1467] Processing CreateTablet for tablet 0bad642a13544a3caa0ca9bcaa4a1257 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.606014 4048 tablet_service.cc:1467] Processing CreateTablet for tablet 0bad642a13544a3caa0ca9bcaa4a1257 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.606091 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.606099 4048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad642a13544a3caa0ca9bcaa4a1257. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.606099 3918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad642a13544a3caa0ca9bcaa4a1257. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.606223 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 8880a7467c6a44da96f98c3cc368c9f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.606300 3916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8880a7467c6a44da96f98c3cc368c9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.607254 4790 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:39.605953 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 87835ddc19954987b0f17f6aa0370c10 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.607386 4796 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:39.607405 4179 tablet_service.cc:1467] Processing CreateTablet for tablet 0bad642a13544a3caa0ca9bcaa4a1257 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.607465 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.607496 4179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad642a13544a3caa0ca9bcaa4a1257. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.607640 4790 tablet_bootstrap.cc:654] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.607717 4796 tablet_bootstrap.cc:654] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.608337 4796 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:39.608366 4796 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:39.608525 4049 tablet_service.cc:1467] Processing CreateTablet for tablet 87835ddc19954987b0f17f6aa0370c10 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.608547 4796 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.608583 4796 raft_consensus.cc:383] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.608595 4796 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.608611 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.608614 4796 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.608709 4796 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.608752 4796 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:39.609143 4796 tablet_bootstrap.cc:654] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.609615 4796 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:39.609668 4796 ts_tablet_manager.cc:1397] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:39.609710 4047 tablet_service.cc:1467] Processing CreateTablet for tablet 8880a7467c6a44da96f98c3cc368c9f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.609776 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8880a7467c6a44da96f98c3cc368c9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.609802 4796 raft_consensus.cc:357] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.609843 4796 raft_consensus.cc:383] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.609858 4796 raft_consensus.cc:738] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.609894 4796 consensus_queue.cc:260] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.609977 4796 ts_tablet_manager.cc:1428] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.610723 3786 tablet_service.cc:1467] Processing CreateTablet for tablet 8880a7467c6a44da96f98c3cc368c9f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.610801 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8880a7467c6a44da96f98c3cc368c9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.611532 4791 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:39.611553 4792 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:39.611853 4792 tablet_bootstrap.cc:654] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.611991 4791 tablet_bootstrap.cc:654] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.612424 4791 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:39.612452 4791 ts_tablet_manager.cc:1397] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:39.612547 4791 raft_consensus.cc:357] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.612607 4791 raft_consensus.cc:383] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.612620 4791 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.612675 4791 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.612785 4791 ts_tablet_manager.cc:1428] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.613703 4790 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:39.613754 4790 ts_tablet_manager.cc:1397] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20250112 21:06:39.613879 4790 raft_consensus.cc:357] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.613955 4790 raft_consensus.cc:383] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.613982 4790 raft_consensus.cc:738] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.614030 4790 consensus_queue.cc:260] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.614120 4790 ts_tablet_manager.cc:1428] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.614187 4790 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:39.614228 4792 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:39.614259 4792 ts_tablet_manager.cc:1397] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20250112 21:06:39.614362 4792 raft_consensus.cc:357] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.614403 4792 raft_consensus.cc:383] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.614415 4792 raft_consensus.cc:738] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.614467 4790 tablet_bootstrap.cc:654] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.614447 4792 consensus_queue.cc:260] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.614523 4792 ts_tablet_manager.cc:1428] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.614563 4792 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:39.614918 4792 tablet_bootstrap.cc:654] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.615010 4790 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:39.615053 4790 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:39.615170 4790 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.615224 4790 raft_consensus.cc:383] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.615247 4790 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.615289 4790 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.615382 4790 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.615427 4790 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:39.615911 4792 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:39.616006 4792 ts_tablet_manager.cc:1397] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:39.616015 4790 tablet_bootstrap.cc:654] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.616222 4792 raft_consensus.cc:357] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.616294 4792 raft_consensus.cc:383] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.616333 4792 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.616374 4792 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.616474 4792 ts_tablet_manager.cc:1428] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.616535 4792 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:39.616744 4790 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:39.616796 4790 ts_tablet_manager.cc:1397] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:39.616919 4790 raft_consensus.cc:357] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.616962 4790 raft_consensus.cc:383] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.616988 4790 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.617040 4790 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.617041 4792 tablet_bootstrap.cc:654] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.617087 4790 ts_tablet_manager.cc:1428] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.617684 4792 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:39.617722 4792 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:39.617828 4792 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.617866 4792 raft_consensus.cc:383] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.617887 4792 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.617934 4792 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.618031 4792 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.735450 4801 raft_consensus.cc:491] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:39.735522 4801 raft_consensus.cc:513] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.735802 4801 leader_election.cc:290] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:39.738323 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad642a13544a3caa0ca9bcaa4a1257" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:39.738322 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad642a13544a3caa0ca9bcaa4a1257" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:39.738401 4200 raft_consensus.cc:2463] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:39.738401 3938 raft_consensus.cc:2463] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:39.738540 4003 leader_election.cc:304] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:39.738663 4801 raft_consensus.cc:2798] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:39.738721 4801 raft_consensus.cc:491] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:39.738744 4801 raft_consensus.cc:3054] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.739261 4801 raft_consensus.cc:513] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.739364 4801 leader_election.cc:290] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:39.739495 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad642a13544a3caa0ca9bcaa4a1257" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:39.739507 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bad642a13544a3caa0ca9bcaa4a1257" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:39.739554 4200 raft_consensus.cc:3054] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.739552 3938 raft_consensus.cc:3054] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.740090 4200 raft_consensus.cc:2463] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 1.
I20250112 21:06:39.740108 3938 raft_consensus.cc:2463] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 1.
I20250112 21:06:39.740247 4003 leader_election.cc:304] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:39.740345 4801 raft_consensus.cc:2798] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:39.740475 4801 raft_consensus.cc:695] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Becoming Leader. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:39.740549 4801 consensus_queue.cc:237] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.741199 3657 catalog_manager.cc:5526] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:39.821702 4813 raft_consensus.cc:491] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:39.821795 4813 raft_consensus.cc:513] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.821971 4813 leader_election.cc:290] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:39.822175 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:39.822257 4069 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:39.822403 4133 leader_election.cc:304] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:39.822507 4813 raft_consensus.cc:2798] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:39.822700 4813 raft_consensus.cc:491] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:39.822747 4813 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.823540 4813 raft_consensus.cc:513] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.823652 4813 leader_election.cc:290] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:39.823777 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:39.823844 4069 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.824352 4069 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:39.824515 4133 leader_election.cc:304] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:39.824594 4813 raft_consensus.cc:2798] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:39.824635 4813 raft_consensus.cc:695] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:39.824703 4813 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:39.824716 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:39.824731 3937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:39.824815 3938 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:39.825273 3657 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:39.832178 4805 raft_consensus.cc:491] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:39.832244 4805 raft_consensus.cc:513] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.832393 4805 leader_election.cc:290] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:39.832584 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:39.832681 3938 raft_consensus.cc:2463] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:39.832825 3741 leader_election.cc:304] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:39.832926 4805 raft_consensus.cc:2798] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:39.832973 4805 raft_consensus.cc:491] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:39.833009 4805 raft_consensus.cc:3054] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.833462 4805 raft_consensus.cc:513] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.833564 4805 leader_election.cc:290] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:39.833710 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:39.833765 3938 raft_consensus.cc:3054] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.834208 3938 raft_consensus.cc:2463] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:39.834345 3741 leader_election.cc:304] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:39.834448 4805 raft_consensus.cc:2798] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:39.834519 4805 raft_consensus.cc:695] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:39.834580 4805 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:39.834780 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:39.834836 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:39.834842 4069 raft_consensus.cc:2463] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:39.835196 3657 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
W20250112 21:06:39.912478 4890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:39.912623 4890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:39.913874 4890 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20250112 21:06:39.944885 3938 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.945084 4068 raft_consensus.cc:3054] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.945138 4817 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.945880 4068 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.946090 3937 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.946188 4817 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.946591 4801 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.946627 4200 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.946954 4801 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.949115 4068 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.949581 3937 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:39.950815 3937 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:39.951087 4813 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:39.951918 4813 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 26.6181 ms
time per row: 0.0133091 ms
I20250112 21:06:39.964553 3664 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:39.966069 3917 tablet_service.cc:1467] Processing CreateTablet for tablet c16b940a9e564da89ec84b46a6cf42d7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.966071 3786 tablet_service.cc:1467] Processing CreateTablet for tablet c16b940a9e564da89ec84b46a6cf42d7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.966185 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16b940a9e564da89ec84b46a6cf42d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.966185 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16b940a9e564da89ec84b46a6cf42d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.966235 4179 tablet_service.cc:1467] Processing CreateTablet for tablet c16b940a9e564da89ec84b46a6cf42d7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.966344 4179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16b940a9e564da89ec84b46a6cf42d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.967383 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 3fae2c2254d04e7a819f5d8638fd97ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.967375 3785 tablet_service.cc:1467] Processing CreateTablet for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.967475 3785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.967475 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fae2c2254d04e7a819f5d8638fd97ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.967540 4180 tablet_service.cc:1467] Processing CreateTablet for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.967633 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.968637 4792 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:39.968848 4796 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:39.968966 4792 tablet_bootstrap.cc:654] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.969250 4796 tablet_bootstrap.cc:654] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.969298 4047 tablet_service.cc:1467] Processing CreateTablet for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.969383 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.970160 4796 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:39.970206 4796 ts_tablet_manager.cc:1397] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:39.970324 4790 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:39.970351 4796 raft_consensus.cc:357] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.970417 4796 raft_consensus.cc:383] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.970441 4796 raft_consensus.cc:738] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.970489 4796 consensus_queue.cc:260] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.970578 4796 ts_tablet_manager.cc:1428] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.970638 4796 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:39.970718 4790 tablet_bootstrap.cc:654] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.970795 4792 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:39.970849 4792 ts_tablet_manager.cc:1397] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:39.970990 4792 raft_consensus.cc:357] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.971052 4792 raft_consensus.cc:383] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.971050 4796 tablet_bootstrap.cc:654] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.971072 4792 raft_consensus.cc:738] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.971101 3785 tablet_service.cc:1467] Processing CreateTablet for tablet 3fae2c2254d04e7a819f5d8638fd97ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.971107 4792 consensus_queue.cc:260] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.971174 3785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fae2c2254d04e7a819f5d8638fd97ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.971189 4792 ts_tablet_manager.cc:1428] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.971372 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 3fae2c2254d04e7a819f5d8638fd97ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:39.971444 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fae2c2254d04e7a819f5d8638fd97ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:39.971670 4791 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:39.972434 4790 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:39.972461 4792 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:39.972473 4790 ts_tablet_manager.cc:1397] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:39.972613 4790 raft_consensus.cc:357] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:39.972751 4790 raft_consensus.cc:383] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.972776 4790 raft_consensus.cc:738] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.972813 4790 consensus_queue.cc:260] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:39.972863 4792 tablet_bootstrap.cc:654] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.973366 4792 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:39.973407 4792 ts_tablet_manager.cc:1397] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:39.973404 4791 tablet_bootstrap.cc:654] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.973516 4792 raft_consensus.cc:357] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.973567 4792 raft_consensus.cc:383] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.973582 4792 raft_consensus.cc:738] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.973618 4792 consensus_queue.cc:260] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.973683 4792 ts_tablet_manager.cc:1428] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.973704 4790 ts_tablet_manager.cc:1428] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:39.974117 4791 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:39.974130 4796 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:39.974169 4791 ts_tablet_manager.cc:1397] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:39.974175 4796 ts_tablet_manager.cc:1397] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250112 21:06:39.974306 4796 raft_consensus.cc:357] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.974299 4791 raft_consensus.cc:357] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.974368 4796 raft_consensus.cc:383] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.974368 4791 raft_consensus.cc:383] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.974391 4796 raft_consensus.cc:738] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.974393 4791 raft_consensus.cc:738] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.974431 4796 consensus_queue.cc:260] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.974437 4791 consensus_queue.cc:260] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.974509 4796 ts_tablet_manager.cc:1428] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.974532 4791 ts_tablet_manager.cc:1428] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.974565 4796 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:39.974586 4791 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:39.974913 4791 tablet_bootstrap.cc:654] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.974960 4796 tablet_bootstrap.cc:654] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.975708 4796 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:39.975761 4796 ts_tablet_manager.cc:1397] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:39.975886 4796 raft_consensus.cc:357] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:39.975940 4796 raft_consensus.cc:383] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.975957 4796 raft_consensus.cc:738] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.975991 4796 consensus_queue.cc:260] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:39.976058 4796 ts_tablet_manager.cc:1428] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.976303 4791 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:39.976343 4791 ts_tablet_manager.cc:1397] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250112 21:06:39.976480 4791 raft_consensus.cc:357] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:39.976536 4791 raft_consensus.cc:383] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.976559 4791 raft_consensus.cc:738] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.976622 4791 consensus_queue.cc:260] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:39.976827 4791 ts_tablet_manager.cc:1428] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:39.976912 4791 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:39.977280 4791 tablet_bootstrap.cc:654] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:39.977798 4791 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:39.977849 4791 ts_tablet_manager.cc:1397] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:39.978036 4791 raft_consensus.cc:357] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.978111 4791 raft_consensus.cc:383] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:39.978125 4791 raft_consensus.cc:738] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:39.978156 4791 consensus_queue.cc:260] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:39.978242 4791 ts_tablet_manager.cc:1428] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:39.998962 4817 raft_consensus.cc:491] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:39.999018 4817 raft_consensus.cc:513] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:39.999145 4817 leader_election.cc:290] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:39.999311 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc53f2e07c4746dcb4b0ddcb8791a4d7" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:39.999397 4200 raft_consensus.cc:2463] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:39.999363 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc53f2e07c4746dcb4b0ddcb8791a4d7" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:39.999435 4069 raft_consensus.cc:2463] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:39.999536 3740 leader_election.cc:304] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:39.999619 4817 raft_consensus.cc:2798] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:39.999650 4817 raft_consensus.cc:491] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:39.999668 4817 raft_consensus.cc:3054] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.000286 4817 raft_consensus.cc:513] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.000375 4817 leader_election.cc:290] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:40.000530 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc53f2e07c4746dcb4b0ddcb8791a4d7" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:40.000568 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc53f2e07c4746dcb4b0ddcb8791a4d7" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:40.000595 4200 raft_consensus.cc:3054] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.000613 4069 raft_consensus.cc:3054] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.001108 4069 raft_consensus.cc:2463] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:40.001129 4200 raft_consensus.cc:2463] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:40.001245 3740 leader_election.cc:304] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:40.001327 4817 raft_consensus.cc:2798] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.001381 4817 raft_consensus.cc:695] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:40.001410 4817 consensus_queue.cc:237] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.001868 3659 catalog_manager.cc:5526] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:40.018738 4797 raft_consensus.cc:491] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:40.018818 4797 raft_consensus.cc:513] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.018977 4797 leader_election.cc:290] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:40.019235 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae2c2254d04e7a819f5d8638fd97ea" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:40.019312 3807 raft_consensus.cc:2463] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:40.019455 3872 leader_election.cc:304] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:40.019567 4797 raft_consensus.cc:2798] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:40.019624 4797 raft_consensus.cc:491] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:40.019657 4797 raft_consensus.cc:3054] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.020216 4797 raft_consensus.cc:513] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.020316 4797 leader_election.cc:290] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:40.020470 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae2c2254d04e7a819f5d8638fd97ea" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:40.020541 3807 raft_consensus.cc:3054] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.021155 3807 raft_consensus.cc:2463] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:40.021317 3872 leader_election.cc:304] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:40.021415 4797 raft_consensus.cc:2798] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.021471 4797 raft_consensus.cc:695] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:40.021531 4797 consensus_queue.cc:237] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.022109 3659 catalog_manager.cc:5526] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:40.022717 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae2c2254d04e7a819f5d8638fd97ea" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:40.022759 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae2c2254d04e7a819f5d8638fd97ea" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:40.022804 4200 raft_consensus.cc:2463] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:40.317834 4800 raft_consensus.cc:491] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:40.317924 4800 raft_consensus.cc:513] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.318094 4800 leader_election.cc:290] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:40.318305 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b940a9e564da89ec84b46a6cf42d7" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:40.318305 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b940a9e564da89ec84b46a6cf42d7" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:40.318403 3807 raft_consensus.cc:2463] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:40.318403 4200 raft_consensus.cc:2463] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:40.318554 3872 leader_election.cc:304] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:40.318681 4800 raft_consensus.cc:2798] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:40.318724 4800 raft_consensus.cc:491] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:40.318745 4800 raft_consensus.cc:3054] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.319438 4800 raft_consensus.cc:513] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.319561 4800 leader_election.cc:290] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:40.319746 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b940a9e564da89ec84b46a6cf42d7" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:40.319736 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b940a9e564da89ec84b46a6cf42d7" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:40.319810 3807 raft_consensus.cc:3054] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.319811 4200 raft_consensus.cc:3054] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.320299 4200 raft_consensus.cc:2463] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:40.320341 3807 raft_consensus.cc:2463] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:40.320470 3871 leader_election.cc:304] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:40.320569 4800 raft_consensus.cc:2798] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.320631 4800 raft_consensus.cc:695] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:40.320690 4800 consensus_queue.cc:237] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.321241 3659 catalog_manager.cc:5526] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:40.392377 4817 consensus_queue.cc:1035] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.393577 4817 consensus_queue.cc:1035] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 21:06:40.484041 4943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:40.484242 4943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:40.485965 4943 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20250112 21:06:40.498379 4200 raft_consensus.cc:3054] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.499356 4800 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.500964 4797 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.517632 4200 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:40.517632 3806 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:40.518132 4797 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.518368 4797 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 22.0158 ms
time per row: 0.0110079 ms
I20250112 21:06:40.533164 3659 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:40.534600 3785 tablet_service.cc:1467] Processing CreateTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.534749 3785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.534700 4049 tablet_service.cc:1467] Processing CreateTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.534716 4047 tablet_service.cc:1467] Processing CreateTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.534749 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.534824 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.534864 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.534933 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.534600 3786 tablet_service.cc:1467] Processing CreateTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.536082 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.536100 4997 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:40.536128 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 74ff25b3286e45af8ddd38965062fca5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.536252 3916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ff25b3286e45af8ddd38965062fca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.536257 4998 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:40.536500 4997 tablet_bootstrap.cc:654] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.536744 4998 tablet_bootstrap.cc:654] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.537357 4998 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:40.537398 4998 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:40.537417 4048 tablet_service.cc:1467] Processing CreateTablet for tablet 74ff25b3286e45af8ddd38965062fca5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.537503 4048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ff25b3286e45af8ddd38965062fca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.537509 4997 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:40.537540 4997 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.537530 4998 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.537593 4998 raft_consensus.cc:383] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.537613 4998 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.537659 4998 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.537643 4997 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.537701 4997 raft_consensus.cc:383] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.537726 4997 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.537751 4998 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.537762 4997 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.537796 4998 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:40.537850 4997 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.537894 4997 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:40.538280 4997 tablet_bootstrap.cc:654] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.538420 4998 tablet_bootstrap.cc:654] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.538641 4180 tablet_service.cc:1467] Processing CreateTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.538722 4997 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:40.538738 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.538763 4997 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.538874 4997 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.538908 4997 raft_consensus.cc:383] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.538920 4997 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.538938 4997 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.538939 4998 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:40.538973 4998 ts_tablet_manager.cc:1397] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:40.539009 4997 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.539088 4998 raft_consensus.cc:357] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.539131 4998 raft_consensus.cc:383] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.539147 4998 raft_consensus.cc:738] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.539188 4998 consensus_queue.cc:260] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.539278 4998 ts_tablet_manager.cc:1428] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.539783 5003 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:40.539789 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 74ff25b3286e45af8ddd38965062fca5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.539983 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ff25b3286e45af8ddd38965062fca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.540200 5003 tablet_bootstrap.cc:654] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.540683 5003 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:40.540731 5003 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.540860 5003 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.540901 5003 raft_consensus.cc:383] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.540915 5003 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.540967 5003 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.541118 5003 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.541177 5003 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:40.541517 5003 tablet_bootstrap.cc:654] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.541549 4996 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:40.542076 4996 tablet_bootstrap.cc:654] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.542099 5003 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:40.542129 5003 ts_tablet_manager.cc:1397] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.542219 5003 raft_consensus.cc:357] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.542264 5003 raft_consensus.cc:383] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.542279 5003 raft_consensus.cc:738] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.542322 5003 consensus_queue.cc:260] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.542510 5003 ts_tablet_manager.cc:1428] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.542563 4996 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:40.542613 4996 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.542747 4996 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.542814 4996 raft_consensus.cc:383] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.542845 4996 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.542889 4996 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.542984 4996 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.543044 4996 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:40.543305 4996 tablet_bootstrap.cc:654] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.543836 4996 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:40.543874 4996 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.543982 4996 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.544027 4996 raft_consensus.cc:383] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.544039 4996 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.544085 4996 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.544166 4996 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.544272 4996 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:40.544648 4996 tablet_bootstrap.cc:654] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.545236 4996 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:40.545272 4996 ts_tablet_manager.cc:1397] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:40.545359 4996 raft_consensus.cc:357] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.545399 4996 raft_consensus.cc:383] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.545411 4996 raft_consensus.cc:738] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.545440 4996 consensus_queue.cc:260] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.545522 4996 ts_tablet_manager.cc:1428] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.689936 4813 raft_consensus.cc:491] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:40.690021 4813 raft_consensus.cc:513] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.690174 4813 leader_election.cc:290] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:40.690372 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ff25b3286e45af8ddd38965062fca5" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:40.690410 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ff25b3286e45af8ddd38965062fca5" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:40.690462 4069 raft_consensus.cc:2463] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:40.690480 3936 raft_consensus.cc:2463] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:40.690644 4134 leader_election.cc:304] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:40.690758 4813 raft_consensus.cc:2798] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:40.690809 4813 raft_consensus.cc:491] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:40.690824 4813 raft_consensus.cc:3054] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.691431 4813 raft_consensus.cc:513] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.691540 4813 leader_election.cc:290] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:40.691692 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ff25b3286e45af8ddd38965062fca5" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:40.691716 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ff25b3286e45af8ddd38965062fca5" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:40.691764 3936 raft_consensus.cc:3054] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.691777 4069 raft_consensus.cc:3054] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.692312 4069 raft_consensus.cc:2463] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:40.692409 3936 raft_consensus.cc:2463] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:40.692472 4133 leader_election.cc:304] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:40.692579 4813 raft_consensus.cc:2798] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.692682 4813 raft_consensus.cc:695] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:40.692756 4813 consensus_queue.cc:237] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:40.693313 3659 catalog_manager.cc:5526] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:40.708106 4813 raft_consensus.cc:491] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:40.708163 4813 raft_consensus.cc:513] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.708253 4813 leader_election.cc:290] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:40.708425 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:40.708427 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:40.708498 3807 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:40.708499 4069 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:40.708635 4133 leader_election.cc:304] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:40.708765 4813 raft_consensus.cc:2798] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:40.708809 4813 raft_consensus.cc:491] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:40.708838 4813 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.709345 4813 raft_consensus.cc:513] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.709463 4813 leader_election.cc:290] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:40.709592 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:40.709648 4069 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.709611 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:40.709676 3807 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.710115 4069 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:40.710254 3807 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:40.710275 4133 leader_election.cc:304] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:40.710397 4813 raft_consensus.cc:2798] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.710445 4813 raft_consensus.cc:695] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:40.710493 4813 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:40.711040 3659 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:40.750702 4988 raft_consensus.cc:491] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:40.750768 4988 raft_consensus.cc:513] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.750888 4988 leader_election.cc:290] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:40.751083 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2ae40fed4b41218aa58c523d7d9785" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:40.751085 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2ae40fed4b41218aa58c523d7d9785" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:40.751142 4069 raft_consensus.cc:2463] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:40.751149 3807 raft_consensus.cc:2463] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:40.751257 3871 leader_election.cc:304] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:40.751346 4988 raft_consensus.cc:2798] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:40.751384 4988 raft_consensus.cc:491] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:40.751396 4988 raft_consensus.cc:3054] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.751946 4988 raft_consensus.cc:513] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.752053 4988 leader_election.cc:290] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:40.752213 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2ae40fed4b41218aa58c523d7d9785" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:40.752233 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d2ae40fed4b41218aa58c523d7d9785" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:40.752272 4069 raft_consensus.cc:3054] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.752285 3807 raft_consensus.cc:3054] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.752808 4069 raft_consensus.cc:2463] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:40.752811 3807 raft_consensus.cc:2463] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:40.752988 3871 leader_election.cc:304] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:40.753093 4988 raft_consensus.cc:2798] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.753151 4988 raft_consensus.cc:695] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:40.753198 4988 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.753660 3659 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
W20250112 21:06:40.837143 5013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:40.837282 5013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:40.838428 5013 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20250112 21:06:40.867437 4069 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 21:06:40.867681 3936 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:40.867750 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:40.867830 4068 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:40.867878 3806 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 21:06:40.867900 4813 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.867965 4069 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:40.868165 4813 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.868286 5009 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.867664 5043 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 21:06:40.868481 4797 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:40.869724 5041 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
INSERT report
rows total: 2000
time total: 23.2948 ms
time per row: 0.0116474 ms
I20250112 21:06:40.885021 3656 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:40.886381 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 345f4bf45767475ca481525f3d26a34a (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.886399 4180 tablet_service.cc:1467] Processing CreateTablet for tablet b6ba48c8dc7e4ee485f85ee94aca256b (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.886462 4048 tablet_service.cc:1467] Processing CreateTablet for tablet 7493a3e50b0e4fce955005b60a713348 (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.886523 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 345f4bf45767475ca481525f3d26a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.886574 4048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7493a3e50b0e4fce955005b60a713348. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.886598 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ba48c8dc7e4ee485f85ee94aca256b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.887456 4996 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:40.887473 4047 tablet_service.cc:1467] Processing CreateTablet for tablet b6ba48c8dc7e4ee485f85ee94aca256b (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.887616 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ba48c8dc7e4ee485f85ee94aca256b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.887694 5003 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:40.887692 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 345f4bf45767475ca481525f3d26a34a (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.887806 3916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 345f4bf45767475ca481525f3d26a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.887818 4996 tablet_bootstrap.cc:654] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.888008 5003 tablet_bootstrap.cc:654] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.888650 5003 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:40.888731 5003 ts_tablet_manager.cc:1397] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.888859 3786 tablet_service.cc:1467] Processing CreateTablet for tablet 345f4bf45767475ca481525f3d26a34a (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.888860 5003 raft_consensus.cc:357] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.888916 5003 raft_consensus.cc:383] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.888940 5003 raft_consensus.cc:738] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.888957 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 345f4bf45767475ca481525f3d26a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.888983 5003 consensus_queue.cc:260] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.889073 5003 ts_tablet_manager.cc:1428] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.889133 5003 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:40.889214 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 7493a3e50b0e4fce955005b60a713348 (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.889295 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7493a3e50b0e4fce955005b60a713348. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.889462 5003 tablet_bootstrap.cc:654] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.889829 3785 tablet_service.cc:1467] Processing CreateTablet for tablet 7493a3e50b0e4fce955005b60a713348 (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.889922 3785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7493a3e50b0e4fce955005b60a713348. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.890035 5003 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:40.890079 5003 ts_tablet_manager.cc:1397] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:40.890129 4998 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:40.890205 5003 raft_consensus.cc:357] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.890265 5003 raft_consensus.cc:383] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.890286 5003 raft_consensus.cc:738] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.890301 3787 tablet_service.cc:1467] Processing CreateTablet for tablet b6ba48c8dc7e4ee485f85ee94aca256b (DEFAULT_TABLE table=rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:40.890314 5003 consensus_queue.cc:260] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.890380 3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ba48c8dc7e4ee485f85ee94aca256b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:40.890414 5003 ts_tablet_manager.cc:1428] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.890426 4998 tablet_bootstrap.cc:654] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.891034 4998 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:40.891078 4998 ts_tablet_manager.cc:1397] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:40.891201 4998 raft_consensus.cc:357] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.891248 4998 raft_consensus.cc:383] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.891270 4998 raft_consensus.cc:738] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.891314 4998 consensus_queue.cc:260] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.891423 4998 ts_tablet_manager.cc:1428] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.891424 4997 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:40.891484 4998 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:40.891489 4996 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:40.891525 4996 ts_tablet_manager.cc:1397] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250112 21:06:40.891659 4996 raft_consensus.cc:357] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.891713 4997 tablet_bootstrap.cc:654] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.891737 4996 raft_consensus.cc:383] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.891758 4996 raft_consensus.cc:738] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.891801 4996 consensus_queue.cc:260] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.891902 4996 ts_tablet_manager.cc:1428] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:40.891944 4998 tablet_bootstrap.cc:654] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.891976 4996 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:40.892519 4996 tablet_bootstrap.cc:654] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.892737 4997 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:40.892783 4997 ts_tablet_manager.cc:1397] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:40.892932 4997 raft_consensus.cc:357] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.893013 4997 raft_consensus.cc:383] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.893038 4997 raft_consensus.cc:738] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.893079 4997 consensus_queue.cc:260] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.893162 4996 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:40.893201 4997 ts_tablet_manager.cc:1428] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.893210 4996 ts_tablet_manager.cc:1397] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:40.893262 4997 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:40.893332 4996 raft_consensus.cc:357] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.893395 4996 raft_consensus.cc:383] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.893424 4996 raft_consensus.cc:738] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.893469 4996 consensus_queue.cc:260] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.893559 4996 ts_tablet_manager.cc:1428] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.893630 4997 tablet_bootstrap.cc:654] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.894105 4997 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:40.894147 4997 ts_tablet_manager.cc:1397] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:40.894238 4998 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:40.894277 4998 ts_tablet_manager.cc:1397] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:40.894256 4997 raft_consensus.cc:357] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.894300 4997 raft_consensus.cc:383] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.894318 4997 raft_consensus.cc:738] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.894348 4997 consensus_queue.cc:260] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.894381 4998 raft_consensus.cc:357] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.894412 4997 ts_tablet_manager.cc:1428] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.894429 4998 raft_consensus.cc:383] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.894447 4998 raft_consensus.cc:738] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.894456 4997 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:40.894485 4998 consensus_queue.cc:260] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:40.894578 4998 ts_tablet_manager.cc:1428] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.894834 4997 tablet_bootstrap.cc:654] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:40.895529 4997 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:40.895576 4997 ts_tablet_manager.cc:1397] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:40.895682 4997 raft_consensus.cc:357] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.895738 4997 raft_consensus.cc:383] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:40.895752 4997 raft_consensus.cc:738] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:40.895799 4997 consensus_queue.cc:260] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.895880 4997 ts_tablet_manager.cc:1428] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:40.932840 5042 raft_consensus.cc:491] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:40.932905 5042 raft_consensus.cc:513] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.933048 5042 leader_election.cc:290] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:40.933158 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba48c8dc7e4ee485f85ee94aca256b" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:40.933189 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba48c8dc7e4ee485f85ee94aca256b" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:40.933230 4067 raft_consensus.cc:2463] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:40.933248 3806 raft_consensus.cc:2463] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:40.933349 4133 leader_election.cc:304] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:40.933456 5042 raft_consensus.cc:2798] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:40.933503 5042 raft_consensus.cc:491] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:40.933518 5042 raft_consensus.cc:3054] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.934207 5042 raft_consensus.cc:513] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.934309 5042 leader_election.cc:290] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:40.934432 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba48c8dc7e4ee485f85ee94aca256b" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:40.934439 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba48c8dc7e4ee485f85ee94aca256b" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:40.934499 3806 raft_consensus.cc:3054] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.934499 4067 raft_consensus.cc:3054] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.935127 4067 raft_consensus.cc:2463] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:40.935216 3806 raft_consensus.cc:2463] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:40.935333 4133 leader_election.cc:304] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:40.935406 5042 raft_consensus.cc:2798] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.935463 5042 raft_consensus.cc:695] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:40.935509 5042 consensus_queue.cc:237] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.936132 3663 catalog_manager.cc:5526] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:40.979611 4984 raft_consensus.cc:491] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:40.979673 4984 raft_consensus.cc:513] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.979838 4984 leader_election.cc:290] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:40.979980 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:40.980055 4200 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:40.980067 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:40.980137 3936 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:40.980177 3740 leader_election.cc:304] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:40.980350 4984 raft_consensus.cc:2798] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:40.980396 4984 raft_consensus.cc:491] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:40.980419 4984 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.981016 4984 raft_consensus.cc:513] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.981156 4984 leader_election.cc:290] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:40.981312 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:40.981323 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:40.981384 4200 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.981384 3936 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:40.981868 4200 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:40.982024 3936 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:40.982131 3740 leader_election.cc:304] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:40.982223 4984 raft_consensus.cc:2798] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:40.982266 4984 raft_consensus.cc:695] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:40.982307 4984 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:40.982818 3663 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:41.084831 4886 raft_consensus.cc:491] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:41.084913 4886 raft_consensus.cc:513] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.085117 4886 leader_election.cc:290] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:41.085489 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7493a3e50b0e4fce955005b60a713348" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:41.085587 3936 raft_consensus.cc:2463] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:41.085722 4003 leader_election.cc:304] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.085832 4886 raft_consensus.cc:2798] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:41.085886 4886 raft_consensus.cc:491] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:41.085911 4886 raft_consensus.cc:3054] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.086539 4886 raft_consensus.cc:513] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.086680 4886 leader_election.cc:290] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:41.086829 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7493a3e50b0e4fce955005b60a713348" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:41.086901 3936 raft_consensus.cc:3054] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.087450 3936 raft_consensus.cc:2463] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 1.
I20250112 21:06:41.087591 4003 leader_election.cc:304] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.087692 4886 raft_consensus.cc:2798] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:41.087800 4886 raft_consensus.cc:695] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Becoming Leader. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:41.087854 4886 consensus_queue.cc:237] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.088372 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7493a3e50b0e4fce955005b60a713348" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:41.088372 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7493a3e50b0e4fce955005b60a713348" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:41.088477 3806 raft_consensus.cc:2463] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:41.088523 3663 catalog_manager.cc:5526] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
W20250112 21:06:41.185048 5066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:41.185184 5066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:41.186421 5066 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20250112 21:06:41.214676 4200 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.214957 4067 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.214928 4817 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.214990 3806 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.215157 5042 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.215269 5042 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.215466 3936 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.216301 3807 raft_consensus.cc:3054] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.217156 3807 raft_consensus.cc:1270] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.218021 3936 raft_consensus.cc:1270] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.218051 4984 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.219324 5064 consensus_queue.cc:1035] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.219467 5064 consensus_queue.cc:1035] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 23.1265 ms
time per row: 0.0115633 ms
I20250112 21:06:41.232342 3663 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:41.233796 4047 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.233814 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 102b9dd051ff4a2a8da8ff4692e56256 (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.233898 3918 tablet_service.cc:1467] Processing CreateTablet for tablet e82a54be68bc4c3ca23a516694e980eb (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.233919 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6bf2e45872401f8245a586b6fe7431. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.233897 3787 tablet_service.cc:1467] Processing CreateTablet for tablet e82a54be68bc4c3ca23a516694e980eb (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.233940 3916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b9dd051ff4a2a8da8ff4692e56256. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.233991 3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82a54be68bc4c3ca23a516694e980eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.234041 3918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82a54be68bc4c3ca23a516694e980eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.235121 4998 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:41.235134 4997 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:41.235155 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.235255 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6bf2e45872401f8245a586b6fe7431. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.235478 4998 tablet_bootstrap.cc:654] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.235602 4997 tablet_bootstrap.cc:654] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.236059 4998 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:41.236114 4998 ts_tablet_manager.cc:1397] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:41.236235 4998 raft_consensus.cc:357] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.236294 4998 raft_consensus.cc:383] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.236317 4998 raft_consensus.cc:738] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.236340 4048 tablet_service.cc:1467] Processing CreateTablet for tablet 102b9dd051ff4a2a8da8ff4692e56256 (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.236368 4998 consensus_queue.cc:260] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.236444 4048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b9dd051ff4a2a8da8ff4692e56256. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.236462 4998 ts_tablet_manager.cc:1428] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.236519 4998 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:41.236769 4996 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:41.237035 4998 tablet_bootstrap.cc:654] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.237183 4996 tablet_bootstrap.cc:654] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.237584 4998 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:41.237623 4998 ts_tablet_manager.cc:1397] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:41.237685 4996 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:41.237715 4996 ts_tablet_manager.cc:1397] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:41.237731 4998 raft_consensus.cc:357] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.237773 4998 raft_consensus.cc:383] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.237788 4998 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.237820 4998 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.237818 4996 raft_consensus.cc:357] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.237854 4996 raft_consensus.cc:383] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.237865 4996 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.237905 4998 ts_tablet_manager.cc:1428] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.237900 4996 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.237954 4998 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:41.237987 4996 ts_tablet_manager.cc:1428] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.238046 4996 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:41.238353 4996 tablet_bootstrap.cc:654] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.238374 4998 tablet_bootstrap.cc:654] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.238633 4997 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:41.238683 4997 ts_tablet_manager.cc:1397] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20250112 21:06:41.238729 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.238811 4997 raft_consensus.cc:357] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.238840 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6bf2e45872401f8245a586b6fe7431. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.238866 4997 raft_consensus.cc:383] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.238888 4997 raft_consensus.cc:738] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.238934 4997 consensus_queue.cc:260] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.238997 4996 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:41.239030 4996 ts_tablet_manager.cc:1397] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:41.239025 4997 ts_tablet_manager.cc:1428] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.239205 4996 raft_consensus.cc:357] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.239240 4996 raft_consensus.cc:383] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.239251 4996 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.239292 4996 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.239367 4996 ts_tablet_manager.cc:1428] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.239764 5003 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:41.239784 4180 tablet_service.cc:1467] Processing CreateTablet for tablet 102b9dd051ff4a2a8da8ff4692e56256 (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.239897 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b9dd051ff4a2a8da8ff4692e56256. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.240173 5003 tablet_bootstrap.cc:654] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.240634 5003 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:41.240697 5003 ts_tablet_manager.cc:1397] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:41.240721 4179 tablet_service.cc:1467] Processing CreateTablet for tablet e82a54be68bc4c3ca23a516694e980eb (DEFAULT_TABLE table=rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.240808 4179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82a54be68bc4c3ca23a516694e980eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.240840 5003 raft_consensus.cc:357] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.240906 5003 raft_consensus.cc:383] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.240934 5003 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.241000 5003 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.241134 5003 ts_tablet_manager.cc:1428] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.241204 5003 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:41.241544 5003 tablet_bootstrap.cc:654] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.242165 5003 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:41.242209 5003 ts_tablet_manager.cc:1397] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:41.242337 5003 raft_consensus.cc:357] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.242384 5003 raft_consensus.cc:383] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.242403 5003 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.242445 5003 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.242522 5003 ts_tablet_manager.cc:1428] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.242579 5003 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:41.242669 4998 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:41.242808 4998 ts_tablet_manager.cc:1397] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250112 21:06:41.242940 4998 raft_consensus.cc:357] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.243002 4998 raft_consensus.cc:383] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.243022 4998 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.243059 4998 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.243145 4998 ts_tablet_manager.cc:1428] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.243146 5003 tablet_bootstrap.cc:654] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.243762 5003 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:41.243809 5003 ts_tablet_manager.cc:1397] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:41.243942 5003 raft_consensus.cc:357] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.243984 5003 raft_consensus.cc:383] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.244005 5003 raft_consensus.cc:738] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.244052 5003 consensus_queue.cc:260] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.244131 5003 ts_tablet_manager.cc:1428] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.347822 4797 raft_consensus.cc:491] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:41.347925 4797 raft_consensus.cc:513] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.348137 4797 leader_election.cc:290] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:41.348322 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82a54be68bc4c3ca23a516694e980eb" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:41.348335 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82a54be68bc4c3ca23a516694e980eb" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:41.348412 4200 raft_consensus.cc:2463] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:41.348412 3807 raft_consensus.cc:2463] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:41.348564 3872 leader_election.cc:304] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.348681 4797 raft_consensus.cc:2798] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:41.348723 4797 raft_consensus.cc:491] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:41.348745 4797 raft_consensus.cc:3054] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.349337 4797 raft_consensus.cc:513] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.349460 4797 leader_election.cc:290] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:41.349653 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82a54be68bc4c3ca23a516694e980eb" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:41.349680 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82a54be68bc4c3ca23a516694e980eb" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:41.349740 3807 raft_consensus.cc:3054] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.349751 4200 raft_consensus.cc:3054] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.350442 3807 raft_consensus.cc:2463] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:41.350445 4200 raft_consensus.cc:2463] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:41.350621 3872 leader_election.cc:304] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.350720 4797 raft_consensus.cc:2798] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:41.350768 4797 raft_consensus.cc:695] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:41.350816 4797 consensus_queue.cc:237] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.351428 3663 catalog_manager.cc:5526] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:41.370875 5103 raft_consensus.cc:491] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:41.370958 5103 raft_consensus.cc:513] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.371132 5103 leader_election.cc:290] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:41.371272 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:41.371281 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:41.371348 3936 raft_consensus.cc:2463] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:41.371356 4200 raft_consensus.cc:2463] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:41.371515 4002 leader_election.cc:304] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:41.371610 5103 raft_consensus.cc:2798] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:41.371656 5103 raft_consensus.cc:491] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:41.371681 5103 raft_consensus.cc:3054] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.372215 5103 raft_consensus.cc:513] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.372318 5103 leader_election.cc:290] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:41.372458 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:41.372460 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:41.372515 3936 raft_consensus.cc:3054] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.372517 4200 raft_consensus.cc:3054] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.373075 4200 raft_consensus.cc:2463] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 1.
I20250112 21:06:41.373265 4002 leader_election.cc:304] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:41.373365 5103 raft_consensus.cc:2798] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:41.373430 5103 raft_consensus.cc:695] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Becoming Leader. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:41.373492 5103 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.374086 3663 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:41.389153 3936 raft_consensus.cc:2463] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 1.
I20250112 21:06:41.532526 4797 raft_consensus.cc:491] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:41.532600 4797 raft_consensus.cc:513] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.532779 4797 leader_election.cc:290] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:41.532989 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:41.532989 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:41.533077 4200 raft_consensus.cc:2463] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:41.533077 4067 raft_consensus.cc:2463] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:41.533239 3871 leader_election.cc:304] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.533345 4797 raft_consensus.cc:2798] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:41.533388 4797 raft_consensus.cc:491] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:41.533407 4797 raft_consensus.cc:3054] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.533957 4797 raft_consensus.cc:513] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.534068 4797 leader_election.cc:290] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:41.534227 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:41.534245 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:41.534300 4200 raft_consensus.cc:3054] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.534294 4067 raft_consensus.cc:3054] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.534798 4200 raft_consensus.cc:2463] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:41.534849 4067 raft_consensus.cc:2463] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:41.534969 3871 leader_election.cc:304] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.535071 4797 raft_consensus.cc:2798] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:41.535130 4797 raft_consensus.cc:695] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:41.535181 4797 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:41.535758 3663 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
W20250112 21:06:41.664410 5116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:41.664553 5116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:41.665750 5116 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20250112 21:06:41.694660 4200 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.695173 4886 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.695241 3807 raft_consensus.cc:1270] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.695379 4200 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.695391 3936 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.695523 4199 raft_consensus.cc:1270] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.695638 4067 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:41.695665 4886 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.695869 4797 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.695989 4797 consensus_queue.cc:1035] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.697189 5043 consensus_queue.cc:1035] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:41.697320 5043 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 21.9051 ms
time per row: 0.0109526 ms
I20250112 21:06:41.710675 3657 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:41.712085 3787 tablet_service.cc:1467] Processing CreateTablet for tablet 56e0dcaf59aa4303aff6810c70092184 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.712114 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 56e0dcaf59aa4303aff6810c70092184 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.712110 3785 tablet_service.cc:1467] Processing CreateTablet for tablet b3b6276ae83948b6b4ea2771d9ce8355 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.712209 3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e0dcaf59aa4303aff6810c70092184. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.712213 3916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e0dcaf59aa4303aff6810c70092184. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.712201 4048 tablet_service.cc:1467] Processing CreateTablet for tablet 56e0dcaf59aa4303aff6810c70092184 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.712260 3786 tablet_service.cc:1467] Processing CreateTablet for tablet c93763a09f7d4460b1874bf328fb06f5 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.712311 4048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e0dcaf59aa4303aff6810c70092184. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.712368 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c93763a09f7d4460b1874bf328fb06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.712456 3785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b6276ae83948b6b4ea2771d9ce8355. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.713243 4996 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:41.713279 4049 tablet_service.cc:1467] Processing CreateTablet for tablet c93763a09f7d4460b1874bf328fb06f5 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.713382 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c93763a09f7d4460b1874bf328fb06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.713495 3917 tablet_service.cc:1467] Processing CreateTablet for tablet c93763a09f7d4460b1874bf328fb06f5 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.713574 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c93763a09f7d4460b1874bf328fb06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.713651 4996 tablet_bootstrap.cc:654] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.713253 4047 tablet_service.cc:1467] Processing CreateTablet for tablet b3b6276ae83948b6b4ea2771d9ce8355 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.713991 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b6276ae83948b6b4ea2771d9ce8355. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.714174 4996 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:41.714207 4996 ts_tablet_manager.cc:1397] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:41.714322 4996 raft_consensus.cc:357] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.714330 4179 tablet_service.cc:1467] Processing CreateTablet for tablet b3b6276ae83948b6b4ea2771d9ce8355 (DEFAULT_TABLE table=rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:41.714380 4996 raft_consensus.cc:383] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.714397 4996 raft_consensus.cc:738] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.714427 4179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b6276ae83948b6b4ea2771d9ce8355. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:41.714432 4996 consensus_queue.cc:260] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.714511 4996 ts_tablet_manager.cc:1428] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.714582 4996 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:41.714826 4998 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:41.715178 4996 tablet_bootstrap.cc:654] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.715265 4998 tablet_bootstrap.cc:654] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.716048 4998 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:41.716094 4998 ts_tablet_manager.cc:1397] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:41.716220 4998 raft_consensus.cc:357] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.716261 4998 raft_consensus.cc:383] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.716284 4998 raft_consensus.cc:738] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.716331 4998 consensus_queue.cc:260] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.716411 4998 ts_tablet_manager.cc:1428] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.716467 4998 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:41.716814 4996 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:41.716854 4996 ts_tablet_manager.cc:1397] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250112 21:06:41.716910 4998 tablet_bootstrap.cc:654] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.717037 4996 raft_consensus.cc:357] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.717106 4996 raft_consensus.cc:383] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.717128 4996 raft_consensus.cc:738] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.717164 4996 consensus_queue.cc:260] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.717257 4996 ts_tablet_manager.cc:1428] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.717592 4998 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:41.717626 4998 ts_tablet_manager.cc:1397] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:41.717733 4998 raft_consensus.cc:357] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.717765 4998 raft_consensus.cc:383] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.717777 4998 raft_consensus.cc:738] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.717806 4998 consensus_queue.cc:260] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.717877 4998 ts_tablet_manager.cc:1428] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.719471 5003 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:41.719780 5003 tablet_bootstrap.cc:654] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.720340 5003 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:41.720396 5003 ts_tablet_manager.cc:1397] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:41.720535 5003 raft_consensus.cc:357] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.720600 5003 raft_consensus.cc:383] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.720641 5003 raft_consensus.cc:738] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.720711 5003 consensus_queue.cc:260] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.720810 5003 ts_tablet_manager.cc:1428] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.721355 4996 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:41.721654 4996 tablet_bootstrap.cc:654] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.723069 4996 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:41.723112 4996 ts_tablet_manager.cc:1397] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:41.723201 4996 raft_consensus.cc:357] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.723239 4996 raft_consensus.cc:383] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.723260 4996 raft_consensus.cc:738] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.723294 4996 consensus_queue.cc:260] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.723380 4996 ts_tablet_manager.cc:1428] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.725476 4997 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:41.725821 4997 tablet_bootstrap.cc:654] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.726223 4997 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:41.726261 4997 ts_tablet_manager.cc:1397] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:41.726356 4997 raft_consensus.cc:357] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.726401 4997 raft_consensus.cc:383] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.726413 4997 raft_consensus.cc:738] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.726449 4997 consensus_queue.cc:260] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.726542 4997 ts_tablet_manager.cc:1428] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.726614 4997 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:41.727036 4997 tablet_bootstrap.cc:654] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.727490 4997 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:41.727532 4997 ts_tablet_manager.cc:1397] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:41.727663 4997 raft_consensus.cc:357] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.727707 4997 raft_consensus.cc:383] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.727739 4997 raft_consensus.cc:738] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.727790 4997 consensus_queue.cc:260] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.727877 4997 ts_tablet_manager.cc:1428] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.727929 4997 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:41.728314 4997 tablet_bootstrap.cc:654] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:41.728781 4997 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:41.728819 4997 ts_tablet_manager.cc:1397] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:41.728928 4997 raft_consensus.cc:357] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.728969 4997 raft_consensus.cc:383] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:41.728988 4997 raft_consensus.cc:738] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:41.729028 4997 consensus_queue.cc:260] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.729103 4997 ts_tablet_manager.cc:1428] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:41.733060 5043 raft_consensus.cc:491] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:41.733112 5043 raft_consensus.cc:513] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.733233 5043 leader_election.cc:290] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:41.733379 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93763a09f7d4460b1874bf328fb06f5" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:41.733444 4067 raft_consensus.cc:2463] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:41.733518 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93763a09f7d4460b1874bf328fb06f5" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:41.733570 3871 leader_election.cc:304] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.733597 3806 raft_consensus.cc:2463] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:41.733642 5043 raft_consensus.cc:2798] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:41.733670 5043 raft_consensus.cc:491] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:41.733688 5043 raft_consensus.cc:3054] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.734170 5043 raft_consensus.cc:513] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.734259 5043 leader_election.cc:290] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:41.734405 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93763a09f7d4460b1874bf328fb06f5" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:41.734462 4067 raft_consensus.cc:3054] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.734442 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93763a09f7d4460b1874bf328fb06f5" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:41.734488 3806 raft_consensus.cc:3054] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.734935 3806 raft_consensus.cc:2463] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:41.734946 4067 raft_consensus.cc:2463] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:41.735123 3872 leader_election.cc:304] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:41.735221 5043 raft_consensus.cc:2798] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:41.735272 5043 raft_consensus.cc:695] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:41.735318 5043 consensus_queue.cc:237] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:41.735791 3663 catalog_manager.cc:5526] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:41.812433 4817 raft_consensus.cc:491] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:41.812522 4817 raft_consensus.cc:513] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.812645 4817 leader_election.cc:290] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:41.812840 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e0dcaf59aa4303aff6810c70092184" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:41.812856 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e0dcaf59aa4303aff6810c70092184" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:41.812925 4067 raft_consensus.cc:2463] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:41.812929 3936 raft_consensus.cc:2463] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:41.813066 3740 leader_election.cc:304] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:41.813179 4817 raft_consensus.cc:2798] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:41.813211 4817 raft_consensus.cc:491] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:41.813237 4817 raft_consensus.cc:3054] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.813846 4817 raft_consensus.cc:513] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.813968 4817 leader_election.cc:290] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:41.814118 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e0dcaf59aa4303aff6810c70092184" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:41.814152 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e0dcaf59aa4303aff6810c70092184" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:41.814218 3936 raft_consensus.cc:3054] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.814222 4067 raft_consensus.cc:3054] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:41.814837 4067 raft_consensus.cc:2463] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:41.814836 3936 raft_consensus.cc:2463] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:41.815021 3740 leader_election.cc:304] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:41.815102 4817 raft_consensus.cc:2798] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:41.815193 4817 raft_consensus.cc:695] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:41.815232 4817 consensus_queue.cc:237] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.815721 3663 catalog_manager.cc:5526] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:41.999490 4817 raft_consensus.cc:491] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:41.999564 4817 raft_consensus.cc:513] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:41.999704 4817 leader_election.cc:290] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:41.999881 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:41.999953 4199 raft_consensus.cc:2463] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:41.999909 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:42.000000 4067 raft_consensus.cc:2463] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:42.000103 3740 leader_election.cc:304] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:42.000206 4817 raft_consensus.cc:2798] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:42.000254 4817 raft_consensus.cc:491] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:42.000285 4817 raft_consensus.cc:3054] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.000890 4817 raft_consensus.cc:513] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.001004 4817 leader_election.cc:290] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.001188 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:42.001263 4199 raft_consensus.cc:3054] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.001343 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:42.001417 4067 raft_consensus.cc:3054] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.001885 4199 raft_consensus.cc:2463] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:42.001890 4067 raft_consensus.cc:2463] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:42.002027 3740 leader_election.cc:304] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:42.002116 4817 raft_consensus.cc:2798] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:42.002166 4817 raft_consensus.cc:695] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:42.002219 4817 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.002758 3663 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
W20250112 21:06:42.141072 5166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:42.141222 5166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:42.142412 5166 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20250112 21:06:42.170797 3806 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.170905 4067 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.171039 5043 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.171146 5145 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.172528 4977 mvcc.cc:204] Tried to move back new op lower bound from 7113588744889786368 to 7113588743108087808. Current Snapshot: MvccSnapshot[applied={T|T < 7113588744889786368}]
I20250112 21:06:42.173259 4067 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 21:06:42.174347 4817 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.174541 4199 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.175076 4067 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.175992 3936 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 21:06:42.176224 5196 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.177872 5195 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250112 21:06:42.178138 5194 mvcc.cc:204] Tried to move back new op lower bound from 7113588744891641856 to 7113588743435313152. Current Snapshot: MvccSnapshot[applied={T|T < 7113588744891641856}]
I20250112 21:06:42.178305 5163 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 22.1773 ms
time per row: 0.0110887 ms
I20250112 21:06:42.184737 5048 mvcc.cc:204] Tried to move back new op lower bound from 7113588744913391616 to 7113588744201474048. Current Snapshot: MvccSnapshot[applied={T|T < 7113588744913391616 or (T in {7113588744913391616})}]
I20250112 21:06:42.189141 3657 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:42.190629 3786 tablet_service.cc:1467] Processing CreateTablet for tablet 1fb3deb52b8c4cceb1814858ea30e94f (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.190644 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 1fb3deb52b8c4cceb1814858ea30e94f (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.190780 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb3deb52b8c4cceb1814858ea30e94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.190780 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb3deb52b8c4cceb1814858ea30e94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.190788 4179 tablet_service.cc:1467] Processing CreateTablet for tablet 1fb3deb52b8c4cceb1814858ea30e94f (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.190827 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 5da4fd48e1f24680babd4f1874b31b5f (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.190900 4179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb3deb52b8c4cceb1814858ea30e94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.190798 4180 tablet_service.cc:1467] Processing CreateTablet for tablet 90f8eabb6a0d4292953e3dec2efec97d (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.191021 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f8eabb6a0d4292953e3dec2efec97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.192098 5003 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:42.192121 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5da4fd48e1f24680babd4f1874b31b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.192246 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 90f8eabb6a0d4292953e3dec2efec97d (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.192253 4049 tablet_service.cc:1467] Processing CreateTablet for tablet 5da4fd48e1f24680babd4f1874b31b5f (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.192342 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5da4fd48e1f24680babd4f1874b31b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.192342 3916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f8eabb6a0d4292953e3dec2efec97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.193113 5003 tablet_bootstrap.cc:654] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.193481 4997 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:42.193517 3786 tablet_service.cc:1467] Processing CreateTablet for tablet 5da4fd48e1f24680babd4f1874b31b5f (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.193575 4996 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:42.193614 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5da4fd48e1f24680babd4f1874b31b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.193980 4996 tablet_bootstrap.cc:654] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.194133 4997 tablet_bootstrap.cc:654] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.194456 4998 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:42.194537 4996 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:42.194584 4996 ts_tablet_manager.cc:1397] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:42.194726 4996 raft_consensus.cc:357] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.194792 4996 raft_consensus.cc:383] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.194818 4996 raft_consensus.cc:738] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.194873 4996 consensus_queue.cc:260] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.194924 4998 tablet_bootstrap.cc:654] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.194988 4996 ts_tablet_manager.cc:1428] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.195025 5003 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:42.195062 5003 ts_tablet_manager.cc:1397] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:42.195199 5003 raft_consensus.cc:357] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.195247 5003 raft_consensus.cc:383] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.195267 5003 raft_consensus.cc:738] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.195294 4047 tablet_service.cc:1467] Processing CreateTablet for tablet 90f8eabb6a0d4292953e3dec2efec97d (DEFAULT_TABLE table=rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.195314 5003 consensus_queue.cc:260] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.195389 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f8eabb6a0d4292953e3dec2efec97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.195420 5003 ts_tablet_manager.cc:1428] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.195482 5003 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:42.195818 5003 tablet_bootstrap.cc:654] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.196571 4996 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:42.196930 4996 tablet_bootstrap.cc:654] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.197038 4998 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:42.197093 4998 ts_tablet_manager.cc:1397] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:42.197235 4998 raft_consensus.cc:357] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.197284 4998 raft_consensus.cc:383] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.197309 4998 raft_consensus.cc:738] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.197362 4998 consensus_queue.cc:260] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.197467 4998 ts_tablet_manager.cc:1428] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.197540 4998 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:42.197813 4996 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:42.197870 4996 ts_tablet_manager.cc:1397] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:42.198024 4996 raft_consensus.cc:357] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.198092 4996 raft_consensus.cc:383] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.198091 4998 tablet_bootstrap.cc:654] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.198114 4996 raft_consensus.cc:738] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.198169 4996 consensus_queue.cc:260] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.198273 4996 ts_tablet_manager.cc:1428] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.198665 4997 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:42.198730 4997 ts_tablet_manager.cc:1397] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250112 21:06:42.198746 4998 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:42.198783 4998 ts_tablet_manager.cc:1397] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:42.198873 4997 raft_consensus.cc:357] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.198932 4997 raft_consensus.cc:383] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.198952 4997 raft_consensus.cc:738] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.198907 4998 raft_consensus.cc:357] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.198972 4998 raft_consensus.cc:383] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.198994 4998 raft_consensus.cc:738] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.198997 4997 consensus_queue.cc:260] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.199034 4998 consensus_queue.cc:260] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.199113 4997 ts_tablet_manager.cc:1428] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.199123 4998 ts_tablet_manager.cc:1428] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.199173 4997 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:42.199496 4997 tablet_bootstrap.cc:654] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.200124 4997 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:42.200176 4997 ts_tablet_manager.cc:1397] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:42.200254 5003 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:42.200304 5003 ts_tablet_manager.cc:1397] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250112 21:06:42.200302 4997 raft_consensus.cc:357] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.200351 4997 raft_consensus.cc:383] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.200371 4997 raft_consensus.cc:738] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.200399 4997 consensus_queue.cc:260] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.200461 5003 raft_consensus.cc:357] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.200488 4997 ts_tablet_manager.cc:1428] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:42.200523 5003 raft_consensus.cc:383] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.200546 5003 raft_consensus.cc:738] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.200599 5003 consensus_queue.cc:260] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.200712 5003 ts_tablet_manager.cc:1428] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:42.200773 5003 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:42.201164 5003 tablet_bootstrap.cc:654] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.201838 5003 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:42.201885 5003 ts_tablet_manager.cc:1397] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:42.202028 5003 raft_consensus.cc:357] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.202076 5003 raft_consensus.cc:383] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.202090 5003 raft_consensus.cc:738] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.202118 5003 consensus_queue.cc:260] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.202188 5003 ts_tablet_manager.cc:1428] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.238320 5145 raft_consensus.cc:491] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:42.238399 5145 raft_consensus.cc:513] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.238525 5145 leader_election.cc:290] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.238655 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f8eabb6a0d4292953e3dec2efec97d" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:42.238687 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f8eabb6a0d4292953e3dec2efec97d" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:42.238739 4199 raft_consensus.cc:2463] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:42.238765 4068 raft_consensus.cc:2463] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:42.238874 3871 leader_election.cc:304] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:42.238991 5145 raft_consensus.cc:2798] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:42.239037 5145 raft_consensus.cc:491] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:42.239058 5145 raft_consensus.cc:3054] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.239604 5145 raft_consensus.cc:513] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.239694 5145 leader_election.cc:290] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.239878 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f8eabb6a0d4292953e3dec2efec97d" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:42.239892 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f8eabb6a0d4292953e3dec2efec97d" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:42.239976 4199 raft_consensus.cc:3054] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.239976 4068 raft_consensus.cc:3054] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.240585 4199 raft_consensus.cc:2463] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:42.240582 4068 raft_consensus.cc:2463] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:42.240765 3871 leader_election.cc:304] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:42.240866 5145 raft_consensus.cc:2798] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:42.240919 5145 raft_consensus.cc:695] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:42.240978 5145 consensus_queue.cc:237] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:42.241497 3657 catalog_manager.cc:5526] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:42.342062 5041 raft_consensus.cc:491] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:42.342159 5041 raft_consensus.cc:513] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.342325 5041 leader_election.cc:290] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.342461 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:42.342530 4068 raft_consensus.cc:2463] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:42.342482 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:42.342592 3807 raft_consensus.cc:2463] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:42.342685 4133 leader_election.cc:304] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:42.342806 5041 raft_consensus.cc:2798] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:42.342849 5041 raft_consensus.cc:491] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:42.342883 5041 raft_consensus.cc:3054] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.343465 5041 raft_consensus.cc:513] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.343583 5041 leader_election.cc:290] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.343787 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:42.343816 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:42.343868 3807 raft_consensus.cc:3054] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.343871 4068 raft_consensus.cc:3054] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.344393 4068 raft_consensus.cc:2463] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:42.344475 3807 raft_consensus.cc:2463] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:42.344615 4133 leader_election.cc:304] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:42.344703 5041 raft_consensus.cc:2798] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:42.344815 5041 raft_consensus.cc:695] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:42.344869 5041 consensus_queue.cc:237] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.345417 3657 catalog_manager.cc:5526] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:42.371268 5219 raft_consensus.cc:491] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:42.371341 5219 raft_consensus.cc:513] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.371488 5219 leader_election.cc:290] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:42.371693 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb3deb52b8c4cceb1814858ea30e94f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:42.371695 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb3deb52b8c4cceb1814858ea30e94f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:42.371776 3807 raft_consensus.cc:2463] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:42.371786 3936 raft_consensus.cc:2463] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:42.371913 4134 leader_election.cc:304] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:42.371994 5219 raft_consensus.cc:2798] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:42.372040 5219 raft_consensus.cc:491] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:42.372059 5219 raft_consensus.cc:3054] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.372783 5219 raft_consensus.cc:513] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.372885 5219 leader_election.cc:290] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:42.373044 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb3deb52b8c4cceb1814858ea30e94f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:42.373046 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fb3deb52b8c4cceb1814858ea30e94f" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:42.373124 3807 raft_consensus.cc:3054] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.373124 3936 raft_consensus.cc:3054] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.373729 3807 raft_consensus.cc:2463] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:42.373754 3936 raft_consensus.cc:2463] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:42.373884 4134 leader_election.cc:304] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:42.373975 5219 raft_consensus.cc:2798] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:42.374032 5219 raft_consensus.cc:695] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:42.374082 5219 consensus_queue.cc:237] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:42.374624 3657 catalog_manager.cc:5526] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
W20250112 21:06:42.492595 5222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:42.492920 5222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:42.494232 5222 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20250112 21:06:42.524703 4068 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.524744 4199 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.524974 4797 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.525123 4797 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.526921 3807 raft_consensus.cc:1270] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.526968 3806 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.526988 3936 raft_consensus.cc:1270] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.527009 4068 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:42.527190 5219 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.527244 5218 consensus_queue.cc:1035] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.527386 5218 consensus_queue.cc:1035] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.527274 5041 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:42.531976 5248 mvcc.cc:204] Tried to move back new op lower bound from 7113588746347847680 to 7113588745724649472. Current Snapshot: MvccSnapshot[applied={T|T < 7113588746347847680}]
INSERT report
rows total: 2000
time total: 25.9039 ms
time per row: 0.012952 ms
I20250112 21:06:42.544847 3661 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:42.546377 3916 tablet_service.cc:1467] Processing CreateTablet for tablet 0fce4710d0784d02991a6465522323d1 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.546397 3787 tablet_service.cc:1467] Processing CreateTablet for tablet 47ca3308b60c4367ab34f25ed5ecb0c0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.546401 3918 tablet_service.cc:1467] Processing CreateTablet for tablet dd01159cf53642739a8a1dbcacbc933d (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.546538 3918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd01159cf53642739a8a1dbcacbc933d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.546535 3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47ca3308b60c4367ab34f25ed5ecb0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.546561 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 47ca3308b60c4367ab34f25ed5ecb0c0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.546595 4047 tablet_service.cc:1467] Processing CreateTablet for tablet 47ca3308b60c4367ab34f25ed5ecb0c0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.546660 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47ca3308b60c4367ab34f25ed5ecb0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.546695 4047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47ca3308b60c4367ab34f25ed5ecb0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.547717 3916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce4710d0784d02991a6465522323d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.547783 4998 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:42.547899 4049 tablet_service.cc:1467] Processing CreateTablet for tablet dd01159cf53642739a8a1dbcacbc933d (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.547981 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd01159cf53642739a8a1dbcacbc933d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.548094 4998 tablet_bootstrap.cc:654] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.548427 4997 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:42.548863 4996 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:42.548961 4997 tablet_bootstrap.cc:654] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.549227 4996 tablet_bootstrap.cc:654] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.549381 4998 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:42.549419 4998 ts_tablet_manager.cc:1397] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20250112 21:06:42.546397 3786 tablet_service.cc:1467] Processing CreateTablet for tablet 0fce4710d0784d02991a6465522323d1 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.549525 4998 raft_consensus.cc:357] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.549563 4998 raft_consensus.cc:383] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.549577 4998 raft_consensus.cc:738] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.549597 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce4710d0784d02991a6465522323d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.549597 4998 consensus_queue.cc:260] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.549701 4998 ts_tablet_manager.cc:1428] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.549769 4998 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:42.549800 4996 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:42.549837 4996 ts_tablet_manager.cc:1397] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:42.550004 4996 raft_consensus.cc:357] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.550072 4996 raft_consensus.cc:383] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.550092 4996 raft_consensus.cc:738] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.550129 4996 consensus_queue.cc:260] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.550201 4998 tablet_bootstrap.cc:654] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.550241 4996 ts_tablet_manager.cc:1428] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.550328 4996 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:42.550769 4998 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:42.550812 4998 ts_tablet_manager.cc:1397] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:42.550856 4996 tablet_bootstrap.cc:654] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.550932 4998 raft_consensus.cc:357] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.550998 4998 raft_consensus.cc:383] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.551023 4998 raft_consensus.cc:738] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.551069 4998 consensus_queue.cc:260] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.551175 4998 ts_tablet_manager.cc:1428] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.551229 4998 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:42.551379 4997 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:42.551434 4997 ts_tablet_manager.cc:1397] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250112 21:06:42.551525 4996 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:42.551560 4996 ts_tablet_manager.cc:1397] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:42.551577 4997 raft_consensus.cc:357] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.551621 4998 tablet_bootstrap.cc:654] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.551656 4997 raft_consensus.cc:383] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.551677 4997 raft_consensus.cc:738] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.551712 4996 raft_consensus.cc:357] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.551721 4997 consensus_queue.cc:260] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.551764 4996 raft_consensus.cc:383] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.551786 4996 raft_consensus.cc:738] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.551851 4997 ts_tablet_manager.cc:1428] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:42.551834 4996 consensus_queue.cc:260] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.551909 4997 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:42.551921 4996 ts_tablet_manager.cc:1428] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.552291 4997 tablet_bootstrap.cc:654] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.552783 4998 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:42.552834 4998 ts_tablet_manager.cc:1397] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:42.552847 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 0fce4710d0784d02991a6465522323d1 (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.552966 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce4710d0784d02991a6465522323d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.553093 4998 raft_consensus.cc:357] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.553153 4998 raft_consensus.cc:383] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.553172 4998 raft_consensus.cc:738] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.553206 4998 consensus_queue.cc:260] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.553277 4998 ts_tablet_manager.cc:1428] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.554004 5003 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:42.554028 4180 tablet_service.cc:1467] Processing CreateTablet for tablet dd01159cf53642739a8a1dbcacbc933d (DEFAULT_TABLE table=rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:42.554113 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd01159cf53642739a8a1dbcacbc933d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:42.554308 5003 tablet_bootstrap.cc:654] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.554821 5003 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:42.554858 5003 ts_tablet_manager.cc:1397] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:42.555001 5003 raft_consensus.cc:357] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.555063 5003 raft_consensus.cc:383] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.555083 5003 raft_consensus.cc:738] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.555130 5003 consensus_queue.cc:260] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.555233 5003 ts_tablet_manager.cc:1428] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.555289 5003 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:42.555696 5003 tablet_bootstrap.cc:654] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:42.556080 4997 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:42.556136 4997 ts_tablet_manager.cc:1397] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250112 21:06:42.556284 4997 raft_consensus.cc:357] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.556350 4997 raft_consensus.cc:383] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.556370 4997 raft_consensus.cc:738] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.556407 4997 consensus_queue.cc:260] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.556449 5003 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:42.556499 5003 ts_tablet_manager.cc:1397] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:42.556500 4997 ts_tablet_manager.cc:1428] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:42.556644 5003 raft_consensus.cc:357] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.556763 5003 raft_consensus.cc:383] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:42.556797 5003 raft_consensus.cc:738] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:42.556840 5003 consensus_queue.cc:260] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.556923 5003 ts_tablet_manager.cc:1428] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:42.561786 5198 raft_consensus.cc:491] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:42.561841 5198 raft_consensus.cc:513] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.561951 5198 leader_election.cc:290] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.562074 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:42.562153 3936 raft_consensus.cc:2463] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:42.562129 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:42.562201 4068 raft_consensus.cc:2463] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:42.562340 3740 leader_election.cc:304] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:42.562443 5198 raft_consensus.cc:2798] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:42.562471 5198 raft_consensus.cc:491] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:42.562484 5198 raft_consensus.cc:3054] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.562976 5198 raft_consensus.cc:513] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.563084 5198 leader_election.cc:290] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.563232 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:42.563228 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:42.563290 3936 raft_consensus.cc:3054] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.563300 4068 raft_consensus.cc:3054] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.563875 4068 raft_consensus.cc:2463] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:42.563917 3936 raft_consensus.cc:2463] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:42.564045 3740 leader_election.cc:304] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:42.564134 5198 raft_consensus.cc:2798] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:42.564213 5198 raft_consensus.cc:695] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:42.564280 5198 consensus_queue.cc:237] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.564828 3664 catalog_manager.cc:5526] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:42.684490 5249 raft_consensus.cc:491] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:42.684573 5249 raft_consensus.cc:513] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.684777 5249 leader_election.cc:290] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.684959 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd01159cf53642739a8a1dbcacbc933d" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:42.684988 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd01159cf53642739a8a1dbcacbc933d" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:42.685058 4068 raft_consensus.cc:2463] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:42.685065 3936 raft_consensus.cc:2463] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:42.685217 4133 leader_election.cc:304] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:42.685307 5249 raft_consensus.cc:2798] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:42.685349 5249 raft_consensus.cc:491] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:42.685369 5249 raft_consensus.cc:3054] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.686028 5249 raft_consensus.cc:513] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.686139 5249 leader_election.cc:290] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:42.686280 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd01159cf53642739a8a1dbcacbc933d" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:42.686287 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd01159cf53642739a8a1dbcacbc933d" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:42.686338 3936 raft_consensus.cc:3054] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.686342 4068 raft_consensus.cc:3054] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.686926 3936 raft_consensus.cc:2463] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:42.686946 4068 raft_consensus.cc:2463] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:42.687104 4133 leader_election.cc:304] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:42.687191 5249 raft_consensus.cc:2798] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:42.687242 5249 raft_consensus.cc:695] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:42.687284 5249 consensus_queue.cc:237] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:42.688011 3664 catalog_manager.cc:5526] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:42.900544 5145 raft_consensus.cc:491] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:42.900620 5145 raft_consensus.cc:513] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.900771 5145 leader_election.cc:290] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:42.900894 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fce4710d0784d02991a6465522323d1" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:42.900933 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fce4710d0784d02991a6465522323d1" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:42.901005 3806 raft_consensus.cc:2463] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:42.901005 4199 raft_consensus.cc:2463] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 0.
I20250112 21:06:42.901135 3871 leader_election.cc:304] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:42.901260 5145 raft_consensus.cc:2798] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:42.901310 5145 raft_consensus.cc:491] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:42.901326 5145 raft_consensus.cc:3054] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.901899 5145 raft_consensus.cc:513] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.902022 5145 leader_election.cc:290] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:42.902159 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fce4710d0784d02991a6465522323d1" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:42.902242 3806 raft_consensus.cc:3054] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.902226 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fce4710d0784d02991a6465522323d1" candidate_uuid: "fd8fa04957924e9f83afd61f9adfb455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:42.902292 4199 raft_consensus.cc:3054] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:42.902819 4199 raft_consensus.cc:2463] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:42.902830 3806 raft_consensus.cc:2463] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd8fa04957924e9f83afd61f9adfb455 in term 1.
I20250112 21:06:42.902979 3872 leader_election.cc:304] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:42.903064 5145 raft_consensus.cc:2798] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:42.903116 5145 raft_consensus.cc:695] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Leader. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:42.903162 5145 consensus_queue.cc:237] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:42.903692 3664 catalog_manager.cc:5526] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66). New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
W20250112 21:06:43.065692 5271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:43.065856 5271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:43.067094 5271 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20250112 21:06:43.092926 5198 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.095431 5198 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 21:06:43.100498 4067 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.100732 5249 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.101567 3936 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.101927 5249 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.103920 3806 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.103992 4199 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.104107 5145 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.104928 4797 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 25.7475 ms
time per row: 0.0128737 ms
I20250112 21:06:43.117370 3658 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58020:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20250112 21:06:43.118806 4049 tablet_service.cc:1467] Processing CreateTablet for tablet 917b9e7ed0ed43efa0a60ff461e4f213 (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.118803 3787 tablet_service.cc:1467] Processing CreateTablet for tablet 7249078cad3e4bc3ab2b62d8887987df (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.118808 3786 tablet_service.cc:1467] Processing CreateTablet for tablet 917b9e7ed0ed43efa0a60ff461e4f213 (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.118942 3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7249078cad3e4bc3ab2b62d8887987df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.118942 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917b9e7ed0ed43efa0a60ff461e4f213. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.118930 4180 tablet_service.cc:1467] Processing CreateTablet for tablet 917b9e7ed0ed43efa0a60ff461e4f213 (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.119024 4180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917b9e7ed0ed43efa0a60ff461e4f213. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.119031 3786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917b9e7ed0ed43efa0a60ff461e4f213. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.120240 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 7249078cad3e4bc3ab2b62d8887987df (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.120277 5336 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:43.120335 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7249078cad3e4bc3ab2b62d8887987df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.120419 5335 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:43.120484 3917 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc2e530f44745948c4607c5f81b1d0e (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.120586 3917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc2e530f44745948c4607c5f81b1d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.120718 5336 tablet_bootstrap.cc:654] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.120741 5335 tablet_bootstrap.cc:654] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.121070 5334 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:43.121419 5334 tablet_bootstrap.cc:654] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.121773 3787 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc2e530f44745948c4607c5f81b1d0e (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.121870 3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc2e530f44745948c4607c5f81b1d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.121977 4178 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc2e530f44745948c4607c5f81b1d0e (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.122025 5334 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:43.122062 4178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc2e530f44745948c4607c5f81b1d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.122077 5334 ts_tablet_manager.cc:1397] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:43.122210 5334 raft_consensus.cc:357] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.122262 5334 raft_consensus.cc:383] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.122279 5334 raft_consensus.cc:738] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.122326 5334 consensus_queue.cc:260] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.122433 5334 ts_tablet_manager.cc:1428] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.122507 5334 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:43.122867 5334 tablet_bootstrap.cc:654] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.123051 4049 tablet_service.cc:1467] Processing CreateTablet for tablet 7249078cad3e4bc3ab2b62d8887987df (DEFAULT_TABLE table=rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20250112 21:06:43.123136 4049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7249078cad3e4bc3ab2b62d8887987df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:43.123478 5339 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:43.123560 5334 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:43.123603 5334 ts_tablet_manager.cc:1397] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:43.123782 5334 raft_consensus.cc:357] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.123840 5334 raft_consensus.cc:383] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.123867 5334 raft_consensus.cc:738] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.123925 5334 consensus_queue.cc:260] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.124042 5334 ts_tablet_manager.cc:1428] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.124109 5334 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:43.124128 5339 tablet_bootstrap.cc:654] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.124536 5334 tablet_bootstrap.cc:654] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.124552 5336 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:43.124596 5336 ts_tablet_manager.cc:1397] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250112 21:06:43.124763 5336 raft_consensus.cc:357] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.124835 5336 raft_consensus.cc:383] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.124861 5336 raft_consensus.cc:738] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.124859 5335 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:43.124903 5335 ts_tablet_manager.cc:1397] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250112 21:06:43.124915 5336 consensus_queue.cc:260] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.125046 5334 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: No bootstrap required, opened a new log
I20250112 21:06:43.125032 5335 raft_consensus.cc:357] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.125061 5336 ts_tablet_manager.cc:1428] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:43.125085 5335 raft_consensus.cc:383] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.125089 5334 ts_tablet_manager.cc:1397] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:43.125108 5335 raft_consensus.cc:738] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.125129 5336 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:43.125154 5335 consensus_queue.cc:260] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.125247 5335 ts_tablet_manager.cc:1428] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.125227 5334 raft_consensus.cc:357] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.125275 5334 raft_consensus.cc:383] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.125294 5334 raft_consensus.cc:738] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.125298 5335 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:43.125355 5334 consensus_queue.cc:260] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.125442 5334 ts_tablet_manager.cc:1428] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.125676 5335 tablet_bootstrap.cc:654] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.126232 5336 tablet_bootstrap.cc:654] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.126461 5339 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455: No bootstrap required, opened a new log
I20250112 21:06:43.126554 5339 ts_tablet_manager.cc:1397] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250112 21:06:43.126801 5339 raft_consensus.cc:357] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.126955 5339 raft_consensus.cc:383] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.126972 5335 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:43.127012 5335 ts_tablet_manager.cc:1397] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:43.127022 5339 raft_consensus.cc:738] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.127151 5335 raft_consensus.cc:357] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.127215 5335 raft_consensus.cc:383] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.127218 5336 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: No bootstrap required, opened a new log
I20250112 21:06:43.127243 5335 raft_consensus.cc:738] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.127252 5336 ts_tablet_manager.cc:1397] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250112 21:06:43.127173 5339 consensus_queue.cc:260] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.127290 5335 consensus_queue.cc:260] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.127378 5335 ts_tablet_manager.cc:1428] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.127375 5336 raft_consensus.cc:357] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.127427 5336 raft_consensus.cc:383] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.127437 5335 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:43.127451 5336 raft_consensus.cc:738] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.127671 5339 ts_tablet_manager.cc:1428] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:43.127497 5336 consensus_queue.cc:260] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.127866 5335 tablet_bootstrap.cc:654] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Neither blocks nor log segments found. Creating new log.
I20250112 21:06:43.128131 5336 ts_tablet_manager.cc:1428] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:43.128563 5335 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: No bootstrap required, opened a new log
I20250112 21:06:43.128611 5335 ts_tablet_manager.cc:1397] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:43.128772 5335 raft_consensus.cc:357] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.128818 5335 raft_consensus.cc:383] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 21:06:43.128850 5335 raft_consensus.cc:738] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:43.128899 5335 consensus_queue.cc:260] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.129006 5335 ts_tablet_manager.cc:1428] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.139866 5249 raft_consensus.cc:491] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:43.139920 5249 raft_consensus.cc:513] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.140061 5249 leader_election.cc:290] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:43.140229 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455" is_pre_election: true
I20250112 21:06:43.140245 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:43.140298 3936 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:43.140306 3806 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 0.
I20250112 21:06:43.140430 4134 leader_election.cc:304] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:43.140519 5249 raft_consensus.cc:2798] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:43.140548 5249 raft_consensus.cc:491] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:43.140581 5249 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.141251 5249 raft_consensus.cc:513] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.141355 5249 leader_election.cc:290] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:43.141503 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:43.141531 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:43.141606 3806 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.141610 3936 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.142120 3806 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:43.142146 3936 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 1.
I20250112 21:06:43.142273 4134 leader_election.cc:304] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:43.142351 5249 raft_consensus.cc:2798] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:43.142418 5249 raft_consensus.cc:695] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:43.142472 5249 consensus_queue.cc:237] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:43.142983 3656 catalog_manager.cc:5526] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:43.160926 5308 raft_consensus.cc:491] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:43.160992 5308 raft_consensus.cc:513] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.161114 5308 leader_election.cc:290] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:43.161254 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7249078cad3e4bc3ab2b62d8887987df" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:43.161278 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7249078cad3e4bc3ab2b62d8887987df" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" is_pre_election: true
I20250112 21:06:43.161321 4199 raft_consensus.cc:2463] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:43.161345 3806 raft_consensus.cc:2463] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 0.
I20250112 21:06:43.161458 4002 leader_election.cc:304] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:43.161535 5308 raft_consensus.cc:2798] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:43.161578 5308 raft_consensus.cc:491] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:43.161608 5308 raft_consensus.cc:3054] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.162088 5308 raft_consensus.cc:513] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.162185 5308 leader_election.cc:290] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:43.162292 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7249078cad3e4bc3ab2b62d8887987df" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:43.162345 4199 raft_consensus.cc:3054] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.162331 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7249078cad3e4bc3ab2b62d8887987df" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:43.162400 3806 raft_consensus.cc:3054] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.162789 4199 raft_consensus.cc:2463] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 1.
I20250112 21:06:43.162879 3806 raft_consensus.cc:2463] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 1.
I20250112 21:06:43.162947 4002 leader_election.cc:304] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 2565f97e9cc54700ab2ce39aa7a546f8; no voters:
I20250112 21:06:43.163028 5308 raft_consensus.cc:2798] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:43.163079 5308 raft_consensus.cc:695] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Becoming Leader. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:43.163118 5308 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.163655 3656 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67). New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:43.226068 5329 raft_consensus.cc:491] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 21:06:43.226145 5329 raft_consensus.cc:513] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.226274 5329 leader_election.cc:290] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:43.226425 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b9e7ed0ed43efa0a60ff461e4f213" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d" is_pre_election: true
I20250112 21:06:43.226461 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b9e7ed0ed43efa0a60ff461e4f213" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" is_pre_election: true
I20250112 21:06:43.226521 4067 raft_consensus.cc:2463] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:43.226513 4199 raft_consensus.cc:2463] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 0.
I20250112 21:06:43.226658 3740 leader_election.cc:304] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:43.226948 5329 raft_consensus.cc:2798] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 21:06:43.227005 5329 raft_consensus.cc:491] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 21:06:43.227034 5329 raft_consensus.cc:3054] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.227643 5329 raft_consensus.cc:513] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.227777 5329 leader_election.cc:290] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:43.227982 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b9e7ed0ed43efa0a60ff461e4f213" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:43.227993 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b9e7ed0ed43efa0a60ff461e4f213" candidate_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:43.228039 4067 raft_consensus.cc:3054] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.228046 4199 raft_consensus.cc:3054] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 0 FOLLOWER]: Advancing to term 1
I20250112 21:06:43.228495 4067 raft_consensus.cc:2463] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:43.228626 4199 raft_consensus.cc:2463] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4e9b42a6de4ff7a2d954e2be23c606 in term 1.
I20250112 21:06:43.228731 3740 leader_election.cc:304] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:43.228840 5329 raft_consensus.cc:2798] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 21:06:43.229094 5329 raft_consensus.cc:695] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Leader. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:43.229168 5329 consensus_queue.cc:237] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:43.229725 3656 catalog_manager.cc:5526] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65). New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
W20250112 21:06:43.309741 5350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:43.309876 5350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:43.311048 5350 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20250112 21:06:43.339648 4067 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.339797 4199 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.339841 3936 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.340178 3806 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.340289 3807 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.340377 5295 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.340602 5218 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.340945 4886 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.341231 5295 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.340642 4199 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 21:06:43.341571 5308 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 21:06:43.341961 5218 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 24.9328 ms
time per row: 0.0124664 ms
I20250112 21:06:43.359324 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.69:40871
--local_ip_for_outbound_sockets=127.26.82.69
--tserver_master_addrs=127.26.82.126:36329
--webserver_port=39055
--webserver_interface=127.26.82.69
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:43.432525 5389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:43.432709 5389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:43.432745 5389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:43.433924 5389 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:43.433964 5389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:43.433984 5389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.69
I20250112 21:06:43.435256 5389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.69:40871
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.82.69
--webserver_port=39055
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.5389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.69
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:43.435458 5389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:43.435672 5389 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:43.440521 5394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:43.441330 5389 server_base.cc:1034] running on GCE node
W20250112 21:06:43.442158 5395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:43.442893 5397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:43.443058 5389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:43.443239 5389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:43.444360 5389 hybrid_clock.cc:648] HybridClock initialized: now 1736716003444342 us; error 29 us; skew 500 ppm
I20250112 21:06:43.445626 5389 webserver.cc:458] Webserver started at http://127.26.82.69:39055/ using document root <none> and password file <none>
I20250112 21:06:43.445789 5389 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:43.445842 5389 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:43.447077 5389 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:43.447572 5403 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.447741 5389 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.447801 5389 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "6f7f47d877a8407185013dba2959fddd"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:43.448062 5389 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:43.489051 5389 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:43.489279 5389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:43.489379 5389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:43.489575 5389 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:43.489930 5389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:43.489977 5389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.490016 5389 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:43.490048 5389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.496510 5389 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.69:40871
I20250112 21:06:43.496547 5516 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.69:40871 every 8 connection(s)
I20250112 21:06:43.496899 5389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250112 21:06:43.501447 5517 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:43.501524 5517 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:43.501720 5517 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:43.502074 3656 ts_manager.cc:194] Re-registered known tserver with Master: 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:43.502417 3656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.69:44897
I20250112 21:06:43.503661 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 5389
I20250112 21:06:43.503927 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.70:43655
--local_ip_for_outbound_sockets=127.26.82.70
--tserver_master_addrs=127.26.82.126:36329
--webserver_port=43459
--webserver_interface=127.26.82.70
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20250112 21:06:43.580505 5520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:43.580644 5520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:43.580693 5520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:43.581930 5520 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:43.581969 5520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:43.582012 5520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.70
I20250112 21:06:43.583490 5520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.70:43655
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.82.70
--webserver_port=43459
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.5520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.70
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:43.583650 5520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:43.583864 5520 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:43.588029 5525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:43.588966 5528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:43.589764 5526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:43.590560 5520 server_base.cc:1034] running on GCE node
I20250112 21:06:43.590684 5520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:43.590869 5520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:43.592008 5520 hybrid_clock.cc:648] HybridClock initialized: now 1736716003591989 us; error 30 us; skew 500 ppm
I20250112 21:06:43.593137 5520 webserver.cc:458] Webserver started at http://127.26.82.70:43459/ using document root <none> and password file <none>
I20250112 21:06:43.593319 5520 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:43.593389 5520 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:43.594497 5520 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:43.595055 5534 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.595201 5520 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.595263 5520 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:43.595520 5520 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:43.607980 5520 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:43.608155 5520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:43.608233 5520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:43.608353 5520 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:43.608551 5520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:43.608575 5520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.608600 5520 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:43.608616 5520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.613891 5520 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.70:43655
I20250112 21:06:43.613945 5647 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.70:43655 every 8 connection(s)
I20250112 21:06:43.614200 5520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20250112 21:06:43.617650 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 5520
I20250112 21:06:43.617985 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.71:37889
--local_ip_for_outbound_sockets=127.26.82.71
--tserver_master_addrs=127.26.82.126:36329
--webserver_port=39637
--webserver_interface=127.26.82.71
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:43.619385 5648 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:43.619470 5648 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:43.619683 5648 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:43.620000 3656 ts_manager.cc:194] Re-registered known tserver with Master: 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:43.620368 3656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.70:42823
W20250112 21:06:43.692850 5651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:43.693001 5651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:43.693030 5651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:43.694208 5651 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:43.694254 5651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:43.694290 5651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.71
I20250112 21:06:43.695626 5651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.71:37889
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.82.71
--webserver_port=39637
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.5651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.71
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:43.695837 5651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:43.696025 5651 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:43.700405 5659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:43.701086 5657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:43.701797 5656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:43.702893 5651 server_base.cc:1034] running on GCE node
I20250112 21:06:43.703054 5651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:43.703224 5651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:43.704351 5651 hybrid_clock.cc:648] HybridClock initialized: now 1736716003704335 us; error 31 us; skew 500 ppm
I20250112 21:06:43.705377 5651 webserver.cc:458] Webserver started at http://127.26.82.71:39637/ using document root <none> and password file <none>
I20250112 21:06:43.705542 5651 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:43.705596 5651 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:43.706648 5651 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:43.707134 5665 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.707286 5651 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.707353 5651 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:43.707593 5651 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:43.724303 5651 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:43.724471 5651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:43.724566 5651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:43.724756 5651 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:43.725011 5651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:43.725041 5651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.725071 5651 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:43.725095 5651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.730684 5651 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.71:37889
I20250112 21:06:43.730737 5778 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.71:37889 every 8 connection(s)
I20250112 21:06:43.731009 5651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20250112 21:06:43.732297 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 5651
I20250112 21:06:43.732601 26953 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskam1Qel/build/release/bin/kudu
/tmp/dist-test-taskam1Qel/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.82.72:37429
--local_ip_for_outbound_sockets=127.26.82.72
--tserver_master_addrs=127.26.82.126:36329
--webserver_port=42879
--webserver_interface=127.26.82.72
--builtin_ntp_servers=127.26.82.84:36673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20250112 21:06:43.735555 5779 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:43.735630 5779 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:43.735822 5779 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:43.736177 3656 ts_manager.cc:194] Re-registered known tserver with Master: 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:43.736764 3656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.71:57513
W20250112 21:06:43.807412 5782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:43.807555 5782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250112 21:06:43.807574 5782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:43.808789 5782 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20250112 21:06:43.808825 5782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250112 21:06:43.808847 5782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.82.72
I20250112 21:06:43.810150 5782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.82.84:36673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.82.72:37429
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.82.72
--webserver_port=42879
--tserver_master_addrs=127.26.82.126:36329
--never_fsync=true
--heap_profile_path=/tmp/kudu.5782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.82.72
--log_dir=/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type RELEASE
built by None at 12 Jan 2025 20:43:15 UTC on a633016ff5c9
build id 4146
I20250112 21:06:43.810303 5782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 21:06:43.810467 5782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 21:06:43.814714 5787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:43.815632 5788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 21:06:43.816370 5790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 21:06:43.817135 5782 server_base.cc:1034] running on GCE node
I20250112 21:06:43.817291 5782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250112 21:06:43.817487 5782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250112 21:06:43.818619 5782 hybrid_clock.cc:648] HybridClock initialized: now 1736716003818600 us; error 28 us; skew 500 ppm
I20250112 21:06:43.819710 5782 webserver.cc:458] Webserver started at http://127.26.82.72:42879/ using document root <none> and password file <none>
I20250112 21:06:43.819890 5782 fs_manager.cc:362] Metadata directory not provided
I20250112 21:06:43.819983 5782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 21:06:43.821141 5782 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:43.821605 5796 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.821739 5782 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.821801 5782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "38c8e8afd3f5411490f56ebf57aee109"
format_stamp: "Formatted at 2025-01-12 21:06:35 on dist-test-slave-929r"
I20250112 21:06:43.822064 5782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 21:06:43.831862 5782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 21:06:43.832026 5782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 21:06:43.832089 5782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 21:06:43.832219 5782 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 21:06:43.832430 5782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 21:06:43.832455 5782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.832473 5782 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 21:06:43.832484 5782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:43.837356 5782 rpc_server.cc:307] RPC server started. Bound to: 127.26.82.72:37429
I20250112 21:06:43.837391 5909 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.82.72:37429 every 8 connection(s)
I20250112 21:06:43.837648 5782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20250112 21:06:43.841809 5910 heartbeater.cc:346] Connected to a master server at 127.26.82.126:36329
I20250112 21:06:43.841900 5910 heartbeater.cc:463] Registering TS with master...
I20250112 21:06:43.842074 5910 heartbeater.cc:510] Master 127.26.82.126:36329 requested a full tablet report, sending...
I20250112 21:06:43.842388 3656 ts_manager.cc:194] Re-registered known tserver with Master: 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:43.842710 3656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.82.72:44993
I20250112 21:06:43.846812 26953 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskam1Qel/build/release/bin/kudu as pid 5782
I20250112 21:06:44.056375 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.063170 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.064272 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.068054 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.070444 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.072228 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.073801 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.079952 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.084172 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.088758 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.092167 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.093997 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.105160 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.105331 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.105535 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.107721 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.110927 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.111111 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.112167 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.116484 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.117902 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.132431 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.132750 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.137195 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.151944 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.155100 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.156042 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.159821 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.167604 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.168249 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.182397 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.184628 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.185508 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.188727 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.189103 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.195333 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.201324 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.201896 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.202155 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.213361 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.235744 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.241779 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.245793 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.245868 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.252957 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.257129 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.265552 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.269076 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.272176 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.273304 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.274060 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.276760 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.278486 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.283917 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.291589 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.292011 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.305625 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.311990 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.316943 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.320097 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.326053 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.330103 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.330729 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.332779 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.338560 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.343102 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.344135 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.344872 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.349151 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.353075 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.356738 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.369174 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.381773 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.386721 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.389772 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.390378 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.396971 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.399511 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.405404 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.409569 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.428092 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.428777 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.429984 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.440539 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.441708 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.443007 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.448796 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.459153 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.480422 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.485343 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.486568 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.495050 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.499555 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.502861 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.506563 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.508929 5517 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:44.510376 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.513357 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.514004 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.514333 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.521586 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.522683 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.524176 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.528767 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.533227 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.539252 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.544544 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.545137 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.551630 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.551803 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.554404 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.565295 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.566001 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.574460 3938 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.575181 3806 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.575371 4068 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.576288 5958 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.576431 5958 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.581916 6635 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.584916 4068 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.584959 3806 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.586494 3658 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.588755 4199 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.589725 4068 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.589779 3938 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.589951 5249 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.590117 5249 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.590564 3873 consensus_peers.cc:487] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c95203cf74b4b90867a9a3a334dbb21. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.592070 3938 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
W20250112 21:06:44.593150 4135 consensus_peers.cc:487] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87835ddc19954987b0f17f6aa0370c10. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.593118 5249 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.594486 4068 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.594734 3664 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.596406 3938 consensus_queue.cc:237] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:44.597014 4199 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.597084 3806 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.597265 5958 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.597416 5958 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.600346 3873 consensus_peers.cc:487] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3fae2c2254d04e7a819f5d8638fd97ea. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.602376 5958 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.603507 3806 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.603574 3664 catalog_manager.cc:5526] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.604095 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.604182 4199 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.606796 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.611999 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.615111 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.615175 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.619839 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.621333 5648 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:44.623332 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.627846 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.635224 4199 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:44.637117 4068 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:44.637161 3938 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:44.637400 6664 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 21:06:44.637778 4135 consensus_peers.cc:487] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87835ddc19954987b0f17f6aa0370c10. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.638288 6664 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:44.640118 6680 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.640265 4068 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
W20250112 21:06:44.641522 4133 consensus_peers.cc:487] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87835ddc19954987b0f17f6aa0370c10. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.641614 3665 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.643067 3938 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.650395 4199 consensus_queue.cc:237] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:44.651042 4068 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.651279 3806 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.652845 6680 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.653133 6680 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.655154 6680 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.655246 3807 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
W20250112 21:06:44.656252 4135 consensus_peers.cc:487] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 979cf479af044430904e610adee50359. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.656446 4068 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.656599 3659 catalog_manager.cc:5526] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.659047 3807 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.659612 4199 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.659658 4068 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.659902 5329 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.660215 6648 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.663969 3742 consensus_peers.cc:487] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3b6276ae83948b6b4ea2771d9ce8355. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.664724 3807 consensus_queue.cc:237] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:44.665174 3938 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.665234 4068 raft_consensus.cc:1270] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.665246 4200 raft_consensus.cc:1270] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:44.665799 3873 consensus_peers.cc:487] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d2ae40fed4b41218aa58c523d7d9785. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.665891 4068 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.666050 3807 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.666074 6634 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.666177 5329 consensus_queue.cc:1035] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.666221 6708 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.666290 5329 consensus_queue.cc:1035] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.667330 6779 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.667418 4199 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.668617 3659 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.668841 4067 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
W20250112 21:06:44.671785 3741 consensus_peers.cc:487] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc53f2e07c4746dcb4b0ddcb8791a4d7. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.671722 6708 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.671945 3807 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.673046 3658 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.673099 4067 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.676957 6648 raft_consensus.cc:2949] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.678131 4067 raft_consensus.cc:2949] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.679013 4200 raft_consensus.cc:2949] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.679347 3661 catalog_manager.cc:5526] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.691211 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.693073 4067 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:44.694744 4200 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.694926 5388 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.695487 3938 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.695690 6868 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.697788 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250112 21:06:44.699221 4003 consensus_peers.cc:487] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 102b9dd051ff4a2a8da8ff4692e56256. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.699272 5388 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.700564 3938 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.701040 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.701602 3663 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.702433 4200 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.706068 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.706190 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.710798 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.715348 3807 consensus_queue.cc:237] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:44.715971 4067 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.715976 3938 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.716337 6779 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.717497 6779 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.718699 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.718953 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.719774 3807 consensus_queue.cc:237] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:44.720381 5329 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.721385 4067 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
W20250112 21:06:44.721599 3740 consensus_peers.cc:487] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47ca3308b60c4367ab34f25ed5ecb0c0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.722657 3938 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.722990 3663 catalog_manager.cc:5526] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.725657 3938 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.725760 4067 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.725833 6806 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.725911 6648 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.728363 6806 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
W20250112 21:06:44.728833 3742 consensus_peers.cc:487] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56e0dcaf59aa4303aff6810c70092184. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.729568 4067 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.730758 3664 catalog_manager.cc:5526] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.733680 3938 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.734376 3936 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:44.734431 4200 consensus_queue.cc:237] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:44.734926 3936 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.735103 6678 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.735466 4133 consensus_peers.cc:487] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd01159cf53642739a8a1dbcacbc933d. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.735769 4067 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.736181 4200 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.736402 6635 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.737432 5779 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:44.737545 6678 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.739864 3871 consensus_peers.cc:487] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e6bf2e45872401f8245a586b6fe7431. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.739899 4067 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.740074 6635 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.740578 6678 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.741755 3936 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.741819 6635 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.741818 4067 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.742763 3661 catalog_manager.cc:5526] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.743549 4068 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.743634 4200 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.745167 3663 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.759934 4067 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.760481 4200 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.760639 6868 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.761059 3807 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.761569 6868 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.762146 4200 consensus_queue.cc:237] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.762615 4068 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:44.762698 4135 consensus_peers.cc:487] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 74ff25b3286e45af8ddd38965062fca5. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.762929 3936 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.762920 6679 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.763106 6679 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.763780 6877 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.765093 3807 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.765728 4200 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.766244 3661 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
W20250112 21:06:44.767314 4004 consensus_peers.cc:487] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7249078cad3e4bc3ab2b62d8887987df. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.767747 6679 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.767782 3807 consensus_queue.cc:237] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
W20250112 21:06:44.768491 3741 consensus_peers.cc:487] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd9176ff75364d81a5b3bdd0e7e10673. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.768764 3936 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.768893 4067 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.768949 6779 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.768976 3936 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.769037 4200 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.769802 3664 catalog_manager.cc:5526] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.770565 6779 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.772147 6779 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.773339 3936 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.773999 4200 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.774562 3661 catalog_manager.cc:5526] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.792748 3936 consensus_queue.cc:237] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:44.793579 4067 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.793650 3807 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.793865 6708 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.794113 3873 consensus_peers.cc:487] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c93763a09f7d4460b1874bf328fb06f5. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.794498 6708 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.797809 6708 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.799384 3807 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.800215 4067 consensus_queue.cc:237] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:44.800786 3663 catalog_manager.cc:5526] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.801947 3936 raft_consensus.cc:1270] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.803325 6868 consensus_queue.cc:1035] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.804288 4067 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
W20250112 21:06:44.804924 4002 consensus_peers.cc:487] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7493a3e50b0e4fce955005b60a713348. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.805424 3807 raft_consensus.cc:1270] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.806126 6868 consensus_queue.cc:1035] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.807090 6868 raft_consensus.cc:2949] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.808754 3936 raft_consensus.cc:2949] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.810331 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.810459 3661 catalog_manager.cc:5526] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.811074 3807 raft_consensus.cc:2949] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.820250 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.828205 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.829293 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.838557 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.840082 3807 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:44.840520 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.842224 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.843334 5910 heartbeater.cc:502] Master 127.26.82.126:36329 was elected leader, sending a full tablet report...
I20250112 21:06:44.844677 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:44.844924 3740 consensus_peers.cc:487] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8880a7467c6a44da96f98c3cc368c9f0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.845135 4067 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.845317 6908 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.845911 6908 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.848526 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.848541 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.851054 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.852278 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.852573 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.854431 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.862275 6908 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.863574 4067 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.863723 3936 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.864951 3663 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.869503 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.870132 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.876360 3936 consensus_queue.cc:237] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
W20250112 21:06:44.877362 3873 consensus_peers.cc:487] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90f8eabb6a0d4292953e3dec2efec97d. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.877501 4200 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.877621 4067 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.877806 6635 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.877916 5958 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.878646 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.880522 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.880757 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:44.885921 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.890093 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:44.896606 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.897424 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:44.898089 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.898458 5958 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.899772 3665 catalog_manager.cc:5526] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.902328 4067 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.904011 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:44.904798 4200 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.920099 4200 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:44.920816 4067 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.921032 6664 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.921183 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.921983 3807 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:44.922542 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:44.922727 3741 consensus_peers.cc:487] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8880a7467c6a44da96f98c3cc368c9f0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.922915 6779 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:44.922937 4067 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:44.923049 3740 consensus_peers.cc:487] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8880a7467c6a44da96f98c3cc368c9f0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.923141 6779 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:44.923543 6664 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.924077 4134 consensus_peers.cc:487] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de6b6ac6eeff431b9ac8cce47ea3a59e. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.924633 5329 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.924808 4067 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.926076 3936 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.926128 3664 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.927795 6679 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.928568 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.929626 4067 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.929661 3665 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.936048 3936 consensus_queue.cc:237] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.936635 3807 raft_consensus.cc:1270] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.936882 4200 raft_consensus.cc:1270] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.936995 4067 consensus_queue.cc:237] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:44.937110 6708 consensus_queue.cc:1035] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.937243 5958 consensus_queue.cc:1035] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.937641 4199 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:44.937803 3873 consensus_peers.cc:487] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e82a54be68bc4c3ca23a516694e980eb. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.937871 6681 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.937932 4003 consensus_peers.cc:487] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0bad642a13544a3caa0ca9bcaa4a1257. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.938055 3936 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.938195 6681 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.938442 5958 raft_consensus.cc:2949] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.939800 3807 raft_consensus.cc:2949] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.940040 4199 raft_consensus.cc:2949] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.940048 3665 catalog_manager.cc:5526] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.941409 6681 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.942526 4199 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.942589 3656 catalog_manager.cc:5526] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.943563 3936 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.956424 3807 consensus_queue.cc:237] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:44.957077 4199 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.957135 4067 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.957248 6908 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.957336 3742 consensus_peers.cc:487] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 917b9e7ed0ed43efa0a60ff461e4f213. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.957516 6908 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.958154 4200 consensus_queue.cc:237] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:44.958743 3936 raft_consensus.cc:1270] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.958743 3806 raft_consensus.cc:1270] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:44.958796 4133 consensus_peers.cc:487] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fb3deb52b8c4cceb1814858ea30e94f. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.958952 6908 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.958994 6680 consensus_queue.cc:1035] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.959014 3807 consensus_queue.cc:237] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:44.959106 6680 consensus_queue.cc:1035] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.959635 3938 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:44.959677 3741 consensus_peers.cc:487] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56e0dcaf59aa4303aff6810c70092184. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.959921 6779 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:44.960126 4068 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.960176 4199 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.961047 4067 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:44.961071 3656 catalog_manager.cc:5526] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
W20250112 21:06:44.961630 3742 consensus_peers.cc:487] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56e0dcaf59aa4303aff6810c70092184. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.961835 6806 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:44.962543 6806 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.963896 3936 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.964103 3663 catalog_manager.cc:5526] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.964111 4067 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:44.966341 6663 raft_consensus.cc:2949] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.966435 3807 raft_consensus.cc:2949] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.967595 3665 catalog_manager.cc:5526] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.967666 3936 raft_consensus.cc:2949] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:44.978235 4199 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:44.978987 4067 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:44.979120 4135 consensus_peers.cc:487] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de6b6ac6eeff431b9ac8cce47ea3a59e. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.979148 6680 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 21:06:44.979254 4134 consensus_peers.cc:487] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de6b6ac6eeff431b9ac8cce47ea3a59e. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.979460 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:44.983789 6680 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:44.984351 6664 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.984454 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.985519 4067 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.985558 3663 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:44.990979 4199 consensus_queue.cc:237] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:44.991606 4067 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.991645 3807 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:44.991945 6679 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:44.992092 6679 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250112 21:06:44.992205 4135 consensus_peers.cc:487] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6ba48c8dc7e4ee485f85ee94aca256b. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.994565 3936 consensus_queue.cc:237] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:44.995061 4067 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:44.995251 3806 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:44.995332 5958 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:44.995410 6708 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 21:06:44.995536 3873 consensus_peers.cc:487] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c93763a09f7d4460b1874bf328fb06f5. This is attempt 1: this message will repeat every 5th retry.
W20250112 21:06:44.995589 3873 consensus_peers.cc:487] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c93763a09f7d4460b1874bf328fb06f5. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:44.996137 5958 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.996249 3806 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.997359 3660 catalog_manager.cc:5526] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.998008 4067 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:44.998761 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.001399 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.002341 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.006259 6679 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.007437 3807 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.007444 4067 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.009588 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.009569 3663 catalog_manager.cc:5526] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.011024 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.019508 3936 consensus_queue.cc:237] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.020359 4199 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.020452 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
W20250112 21:06:45.020599 3873 consensus_peers.cc:487] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90f8eabb6a0d4292953e3dec2efec97d. This is attempt 1: this message will repeat every 5th retry.
W20250112 21:06:45.020661 3873 consensus_peers.cc:487] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90f8eabb6a0d4292953e3dec2efec97d. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.020768 5958 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.021418 4067 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.021615 5958 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.022151 5958 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.023422 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.023368 3660 catalog_manager.cc:5526] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.023936 4199 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.027765 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.028960 4067 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.037317 3807 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
W20250112 21:06:45.038002 3742 consensus_peers.cc:487] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3b6276ae83948b6b4ea2771d9ce8355. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.038295 4067 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.038327 4199 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:45.038565 3740 consensus_peers.cc:487] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3b6276ae83948b6b4ea2771d9ce8355. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.038568 6648 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.038669 6779 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.040031 6779 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.041167 3807 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.041441 4199 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.041896 3936 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:45.042467 4200 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:45.042815 4067 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
W20250112 21:06:45.042906 3742 consensus_peers.cc:487] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 345f4bf45767475ca481525f3d26a34a. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.043098 3660 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.043884 6648 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.044865 6779 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.047880 6779 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.049813 3936 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.049999 3660 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.049338 4200 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.052518 7506 ts_tablet_manager.cc:927] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.052711 7506 tablet_copy_client.cc:323] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.056942 4200 consensus_queue.cc:237] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.057415 3807 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:45.057467 4135 consensus_peers.cc:487] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6ba48c8dc7e4ee485f85ee94aca256b. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.057516 4067 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.057588 6679 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.058766 6679 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 21:06:45.059494 4134 consensus_peers.cc:487] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6ba48c8dc7e4ee485f85ee94aca256b. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.059525 6679 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.059693 3807 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.061712 4067 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.061755 3657 catalog_manager.cc:5526] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.062247 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 3c95203cf74b4b90867a9a3a334dbb21 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49463)
I20250112 21:06:45.062319 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 3c95203cf74b4b90867a9a3a334dbb21 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49463: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-3c95203cf74b4b90867a9a3a334dbb21
I20250112 21:06:45.063098 3958 tablet_copy_source_session.cc:215] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.064085 7506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c95203cf74b4b90867a9a3a334dbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.067600 7506 tablet_copy_client.cc:806] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.067742 7506 tablet_copy_client.cc:670] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.068042 3807 consensus_queue.cc:237] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.068315 7533 ts_tablet_manager.cc:927] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.068600 4067 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:45.068651 3742 consensus_peers.cc:487] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 917b9e7ed0ed43efa0a60ff461e4f213. This is attempt 1: this message will repeat every 5th retry.
W20250112 21:06:45.068751 3742 consensus_peers.cc:487] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 917b9e7ed0ed43efa0a60ff461e4f213. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.068759 4199 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.069239 6908 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.069337 6648 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.070084 7533 tablet_copy_client.cc:323] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.070457 6908 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.070640 4199 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.072454 3659 catalog_manager.cc:5526] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.073073 4067 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.073984 7506 tablet_copy_client.cc:538] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.075009 7506 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.075287 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.077342 7506 log.cc:826] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:45.081136 7553 ts_tablet_manager.cc:927] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.081349 7553 tablet_copy_client.cc:323] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.083043 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.083258 7506 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.083472 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 87835ddc19954987b0f17f6aa0370c10 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:48269)
I20250112 21:06:45.083504 7506 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.083536 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 87835ddc19954987b0f17f6aa0370c10 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:48269: session id = 67a5a7038ced40ddb6f45fdbfd99825c-87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:45.083608 7506 ts_tablet_manager.cc:1397] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20250112 21:06:45.084162 4220 tablet_copy_source_session.cc:215] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.084672 7506 raft_consensus.cc:357] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.084825 7506 raft_consensus.cc:738] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.084910 7506 consensus_queue.cc:260] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.085203 7506 ts_tablet_manager.cc:1428] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250112 21:06:45.087209 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-3c95203cf74b4b90867a9a3a334dbb21 received from {username='slave'} at 127.26.82.71:49463
I20250112 21:06:45.087271 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-3c95203cf74b4b90867a9a3a334dbb21 on tablet 3c95203cf74b4b90867a9a3a334dbb21 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.088133 7533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.089195 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.091507 7533 tablet_copy_client.cc:806] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.091640 7533 tablet_copy_client.cc:670] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.092352 7533 tablet_copy_client.cc:538] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.093140 7533 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.094890 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.101019 7533 log.cc:826] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:45.101482 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.102223 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 3fae2c2254d04e7a819f5d8638fd97ea from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:53179)
I20250112 21:06:45.102288 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 3fae2c2254d04e7a819f5d8638fd97ea from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:53179: session id = 6f7f47d877a8407185013dba2959fddd-3fae2c2254d04e7a819f5d8638fd97ea
I20250112 21:06:45.102811 3958 tablet_copy_source_session.cc:215] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.103503 7553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fae2c2254d04e7a819f5d8638fd97ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.105110 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.106457 7553 tablet_copy_client.cc:806] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.106592 7553 tablet_copy_client.cc:670] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.107254 7553 tablet_copy_client.cc:538] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.107998 7533 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.107991 7553 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.108204 7533 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.108314 7533 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20250112 21:06:45.109933 7553 log.cc:826] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:45.109848 7533 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.110069 7533 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.110201 7533 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.110458 7533 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20250112 21:06:45.110546 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.111248 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-87835ddc19954987b0f17f6aa0370c10 received from {username='slave'} at 127.26.82.70:48269
I20250112 21:06:45.111304 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-87835ddc19954987b0f17f6aa0370c10 on tablet 87835ddc19954987b0f17f6aa0370c10 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.112906 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.118348 4199 consensus_queue.cc:237] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.118893 3807 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:45.119184 6678 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.119215 4067 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:45.119411 4134 consensus_peers.cc:487] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5da4fd48e1f24680babd4f1874b31b5f. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.119482 6680 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.121135 7533 ts_tablet_manager.cc:927] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.121299 7533 tablet_copy_client.cc:323] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.121537 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet dd01159cf53642739a8a1dbcacbc933d from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:48269)
I20250112 21:06:45.121585 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet dd01159cf53642739a8a1dbcacbc933d from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:48269: session id = 67a5a7038ced40ddb6f45fdbfd99825c-dd01159cf53642739a8a1dbcacbc933d
I20250112 21:06:45.122000 7553 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.122069 4220 tablet_copy_source_session.cc:215] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.122188 7553 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.122387 7533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd01159cf53642739a8a1dbcacbc933d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.122421 7553 ts_tablet_manager.cc:1397] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.001s
I20250112 21:06:45.123586 7533 tablet_copy_client.cc:806] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.123692 7533 tablet_copy_client.cc:670] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.123701 7553 raft_consensus.cc:357] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.123944 7553 raft_consensus.cc:738] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.124059 7553 consensus_queue.cc:260] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.124265 7553 ts_tablet_manager.cc:1428] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250112 21:06:45.124449 7533 tablet_copy_client.cc:538] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.125109 7533 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.126856 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-3fae2c2254d04e7a819f5d8638fd97ea received from {username='slave'} at 127.26.82.69:53179
I20250112 21:06:45.126930 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-3fae2c2254d04e7a819f5d8638fd97ea on tablet 3fae2c2254d04e7a819f5d8638fd97ea with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.127748 7594 ts_tablet_manager.cc:927] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:45.127944 7594 tablet_copy_client.cc:323] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:45.130414 7533 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.130618 7533 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.130721 7533 ts_tablet_manager.cc:1397] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250112 21:06:45.130832 7533 raft_consensus.cc:357] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.130910 7533 raft_consensus.cc:738] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.130959 7533 consensus_queue.cc:260] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.131049 7533 ts_tablet_manager.cc:1428] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.131217 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-dd01159cf53642739a8a1dbcacbc933d received from {username='slave'} at 127.26.82.70:48269
I20250112 21:06:45.131269 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-dd01159cf53642739a8a1dbcacbc933d on tablet dd01159cf53642739a8a1dbcacbc933d with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.132078 6680 raft_consensus.cc:2949] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.132890 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:54097)
I20250112 21:06:45.132962 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:54097: session id = 38c8e8afd3f5411490f56ebf57aee109-102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:45.133414 3807 raft_consensus.cc:2949] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.133692 4089 tablet_copy_source_session.cc:215] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.134208 3659 catalog_manager.cc:5526] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.135054 7594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b9dd051ff4a2a8da8ff4692e56256. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.135253 4067 raft_consensus.cc:2949] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.141628 7594 tablet_copy_client.cc:806] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.141770 7594 tablet_copy_client.cc:670] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.142494 7594 tablet_copy_client.cc:538] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.143322 7594 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.145025 7594 log.cc:826] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: Log is configured to *not* fsync() on all Append() calls
I20250112 21:06:45.149240 4199 consensus_queue.cc:237] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.149869 3807 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:45.149910 3936 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:45.150059 4134 consensus_peers.cc:487] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7fc2e530f44745948c4607c5f81b1d0e. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.150141 5249 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.150260 5249 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.150960 7594 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.151257 7594 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.151404 7594 ts_tablet_manager.cc:1397] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20250112 21:06:45.151880 4067 consensus_queue.cc:237] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.152577 3936 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.152499 7594 raft_consensus.cc:357] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.152639 7594 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.152776 7594 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.152834 6877 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.153036 7594 ts_tablet_manager.cc:1428] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20250112 21:06:45.153174 4003 consensus_peers.cc:487] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0bad642a13544a3caa0ca9bcaa4a1257. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.153267 4199 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.153439 6877 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.153779 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-102b9dd051ff4a2a8da8ff4692e56256 received from {username='slave'} at 127.26.82.72:54097
I20250112 21:06:45.153842 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-102b9dd051ff4a2a8da8ff4692e56256 on tablet 102b9dd051ff4a2a8da8ff4692e56256 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.154001 6876 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.154282 6663 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.154907 3936 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.155050 4199 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.155433 3657 catalog_manager.cc:5526] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
W20250112 21:06:45.156123 4004 consensus_peers.cc:487] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0bad642a13544a3caa0ca9bcaa4a1257. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.156085 3659 catalog_manager.cc:5526] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.156392 3807 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.156765 3936 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.172472 3807 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.173008 3936 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.173179 5329 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.173243 4199 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:45.173756 3742 consensus_peers.cc:487] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 345f4bf45767475ca481525f3d26a34a. This is attempt 1: this message will repeat every 5th retry.
W20250112 21:06:45.173821 3742 consensus_peers.cc:487] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 345f4bf45767475ca481525f3d26a34a. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.173913 5329 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.175071 3936 consensus_queue.cc:237] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.175213 5329 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.175513 3807 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20250112 21:06:45.175601 3873 consensus_peers.cc:487] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fce4710d0784d02991a6465522323d1. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.175683 6708 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.175771 4199 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.176421 4200 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:45.176671 7419 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.177191 3664 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.177486 3936 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.182045 7419 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.183326 4199 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.183770 3807 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.185221 3660 catalog_manager.cc:5526] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index -1 to 4, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:45.189121 4067 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.189970 3807 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:45.190213 4004 consensus_peers.cc:487] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7249078cad3e4bc3ab2b62d8887987df. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.190454 4199 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.190696 6876 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 21:06:45.190944 4003 consensus_peers.cc:487] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7249078cad3e4bc3ab2b62d8887987df. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.191123 6876 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.192960 6876 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.194113 3807 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.194543 3660 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.196700 4199 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.202869 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.208369 7533 ts_tablet_manager.cc:927] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.208519 7533 tablet_copy_client.cc:323] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.211140 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.211431 7553 ts_tablet_manager.cc:927] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.211570 7553 tablet_copy_client.cc:323] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.211869 3827 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 47ca3308b60c4367ab34f25ed5ecb0c0 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:50395)
I20250112 21:06:45.211941 3827 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 47ca3308b60c4367ab34f25ed5ecb0c0 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:50395: session id = 67a5a7038ced40ddb6f45fdbfd99825c-47ca3308b60c4367ab34f25ed5ecb0c0
I20250112 21:06:45.212919 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.213150 3827 tablet_copy_source_session.cc:215] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.213245 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.213358 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.214250 7533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47ca3308b60c4367ab34f25ed5ecb0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.214612 7506 ts_tablet_manager.cc:927] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.214722 7506 tablet_copy_client.cc:323] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.215559 7533 tablet_copy_client.cc:806] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.215658 7533 tablet_copy_client.cc:670] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.215746 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.216639 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.216723 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.217432 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.217633 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 979cf479af044430904e610adee50359 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:47595)
I20250112 21:06:45.217681 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 979cf479af044430904e610adee50359 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:47595: session id = 6f7f47d877a8407185013dba2959fddd-979cf479af044430904e610adee50359
I20250112 21:06:45.218138 4220 tablet_copy_source_session.cc:215] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.218461 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.218958 7553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979cf479af044430904e610adee50359. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.220094 7553 tablet_copy_client.cc:806] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.220196 7553 tablet_copy_client.cc:670] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.222678 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 87835ddc19954987b0f17f6aa0370c10 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49797)
I20250112 21:06:45.222728 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 87835ddc19954987b0f17f6aa0370c10 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49797: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:45.223138 4220 tablet_copy_source_session.cc:215] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.223392 7506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.224560 7506 tablet_copy_client.cc:806] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.224651 7506 tablet_copy_client.cc:670] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.227293 7594 ts_tablet_manager.cc:927] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.227443 7594 tablet_copy_client.cc:323] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.228863 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.230516 3827 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:52565)
I20250112 21:06:45.230574 3827 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:52565: session id = 38c8e8afd3f5411490f56ebf57aee109-bc53f2e07c4746dcb4b0ddcb8791a4d7
I20250112 21:06:45.231015 3827 tablet_copy_source_session.cc:215] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.231350 7594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.232470 7594 tablet_copy_client.cc:806] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.232565 7594 tablet_copy_client.cc:670] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.235734 7553 tablet_copy_client.cc:538] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.236464 7553 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.240240 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.241223 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.244324 4199 consensus_queue.cc:237] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
W20250112 21:06:45.244995 4133 consensus_peers.cc:487] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7fc2e530f44745948c4607c5f81b1d0e. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.245352 3807 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.245599 3936 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20250112 21:06:45.245870 4134 consensus_peers.cc:487] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7fc2e530f44745948c4607c5f81b1d0e. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.245959 6664 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.246062 6664 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.247107 5249 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.247808 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.248379 3660 catalog_manager.cc:5526] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.248453 3807 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.248723 3936 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.251596 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.253469 7788 ts_tablet_manager.cc:927] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.253877 7794 ts_tablet_manager.cc:927] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.254125 7794 tablet_copy_client.cc:323] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.254124 7506 tablet_copy_client.cc:538] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.254439 7788 tablet_copy_client.cc:323] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.254667 7798 ts_tablet_manager.cc:927] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.254942 7506 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.255044 7798 tablet_copy_client.cc:323] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.255306 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49463)
I20250112 21:06:45.255359 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49463: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:45.255800 3958 tablet_copy_source_session.cc:215] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.256079 7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.256788 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet dd9176ff75364d81a5b3bdd0e7e10673 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:52565)
I20250112 21:06:45.256843 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet dd9176ff75364d81a5b3bdd0e7e10673 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:52565: session id = 38c8e8afd3f5411490f56ebf57aee109-dd9176ff75364d81a5b3bdd0e7e10673
I20250112 21:06:45.257222 3826 tablet_copy_source_session.cc:215] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.257606 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd9176ff75364d81a5b3bdd0e7e10673. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.257650 7798 tablet_copy_client.cc:806] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.257766 7798 tablet_copy_client.cc:670] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.258213 7798 tablet_copy_client.cc:538] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.258829 7788 tablet_copy_client.cc:806] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.258925 7788 tablet_copy_client.cc:670] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.258976 7594 tablet_copy_client.cc:538] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.259194 7798 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.259586 7594 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.261116 7506 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.261315 7506 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.261436 7506 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20250112 21:06:45.261569 7506 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.261643 7506 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.261708 7506 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.261917 7506 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:45.262091 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-87835ddc19954987b0f17f6aa0370c10 received from {username='slave'} at 127.26.82.71:49797
I20250112 21:06:45.262136 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-87835ddc19954987b0f17f6aa0370c10 on tablet 87835ddc19954987b0f17f6aa0370c10 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.262522 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 2e6bf2e45872401f8245a586b6fe7431 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:38803)
I20250112 21:06:45.262568 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 2e6bf2e45872401f8245a586b6fe7431 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:38803: session id = 67a5a7038ced40ddb6f45fdbfd99825c-2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:45.263427 3958 tablet_copy_source_session.cc:215] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.263748 7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6bf2e45872401f8245a586b6fe7431. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.263929 7594 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.264144 7594 tablet_bootstrap.cc:492] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.264228 7594 ts_tablet_manager.cc:1397] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20250112 21:06:45.264326 7594 raft_consensus.cc:357] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.264389 7594 raft_consensus.cc:738] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.264438 7594 consensus_queue.cc:260] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.264524 7594 ts_tablet_manager.cc:1428] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.264977 7794 tablet_copy_client.cc:806] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.265065 7794 tablet_copy_client.cc:670] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.265350 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-bc53f2e07c4746dcb4b0ddcb8791a4d7 received from {username='slave'} at 127.26.82.72:52565
I20250112 21:06:45.265400 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-bc53f2e07c4746dcb4b0ddcb8791a4d7 on tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.265803 7794 tablet_copy_client.cc:538] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.265851 7788 tablet_copy_client.cc:538] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.266311 7798 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.266448 7788 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.266454 7794 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.266510 7798 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.266598 7798 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250112 21:06:45.266711 7798 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.266793 7798 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.266870 7798 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.267102 7798 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.002s sys 0.000s
I20250112 21:06:45.267412 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-0d2ae40fed4b41218aa58c523d7d9785 received from {username='slave'} at 127.26.82.71:49463
I20250112 21:06:45.267457 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-0d2ae40fed4b41218aa58c523d7d9785 on tablet 0d2ae40fed4b41218aa58c523d7d9785 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.270211 7794 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.270395 7794 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.270486 7794 ts_tablet_manager.cc:1397] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250112 21:06:45.270618 7794 raft_consensus.cc:357] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.270673 7794 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.270733 7794 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.270771 7788 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.270931 7794 ts_tablet_manager.cc:1428] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.002s sys 0.000s
I20250112 21:06:45.270973 7788 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.271059 7788 ts_tablet_manager.cc:1397] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250112 21:06:45.271096 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-2e6bf2e45872401f8245a586b6fe7431 received from {username='slave'} at 127.26.82.70:38803
I20250112 21:06:45.271136 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-2e6bf2e45872401f8245a586b6fe7431 on tablet 2e6bf2e45872401f8245a586b6fe7431 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.271163 7788 raft_consensus.cc:357] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.271219 7788 raft_consensus.cc:738] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.271261 7788 consensus_queue.cc:260] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.271445 7788 ts_tablet_manager.cc:1428] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.271678 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-dd9176ff75364d81a5b3bdd0e7e10673 received from {username='slave'} at 127.26.82.72:52565
I20250112 21:06:45.271735 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-dd9176ff75364d81a5b3bdd0e7e10673 on tablet dd9176ff75364d81a5b3bdd0e7e10673 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.272743 7533 tablet_copy_client.cc:538] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.272945 3936 consensus_queue.cc:237] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.273499 7533 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.273722 3807 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:45.273753 4199 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 21:06:45.274008 7420 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.274200 7420 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 21:06:45.276113 7553 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.276304 7553 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.276400 7553 ts_tablet_manager.cc:1397] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.002s
I20250112 21:06:45.276530 7553 raft_consensus.cc:357] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.276610 7553 raft_consensus.cc:738] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.276695 7533 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.276695 7553 consensus_queue.cc:260] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.276895 7533 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.276932 7553 ts_tablet_manager.cc:1428] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.000s user 0.001s sys 0.003s
I20250112 21:06:45.276988 7533 ts_tablet_manager.cc:1397] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20250112 21:06:45.277096 7533 raft_consensus.cc:357] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.277153 7533 raft_consensus.cc:738] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.277199 7533 consensus_queue.cc:260] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.277294 7533 ts_tablet_manager.cc:1428] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.277807 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-979cf479af044430904e610adee50359 received from {username='slave'} at 127.26.82.69:47595
I20250112 21:06:45.277858 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-979cf479af044430904e610adee50359 on tablet 979cf479af044430904e610adee50359 with peer 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.278206 3871 consensus_peers.cc:487] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c16b940a9e564da89ec84b46a6cf42d7. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.278291 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-47ca3308b60c4367ab34f25ed5ecb0c0 received from {username='slave'} at 127.26.82.70:50395
I20250112 21:06:45.278327 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-47ca3308b60c4367ab34f25ed5ecb0c0 on tablet 47ca3308b60c4367ab34f25ed5ecb0c0 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.279484 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.281337 7420 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.282639 4199 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.283869 3807 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.285318 3660 catalog_manager.cc:5526] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index -1 to 4, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:45.294818 3936 consensus_queue.cc:237] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
W20250112 21:06:45.295531 3873 consensus_peers.cc:487] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fce4710d0784d02991a6465522323d1. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.295625 3807 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.295723 4199 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.295923 6634 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.296089 6634 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.296869 6634 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.298022 4199 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.298143 3807 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:45.298995 3660 catalog_manager.cc:5526] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 4 to 5, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
W20250112 21:06:45.303893 3872 consensus_peers.cc:487] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fce4710d0784d02991a6465522323d1. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.312305 3936 consensus_queue.cc:237] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.312777 3807 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.312932 6635 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 21:06:45.313038 3871 consensus_peers.cc:487] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c16b940a9e564da89ec84b46a6cf42d7. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.313158 4199 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.313418 6635 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 21:06:45.313520 3873 consensus_peers.cc:487] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c16b940a9e564da89ec84b46a6cf42d7. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:45.314008 6635 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.315408 3807 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.315472 4199 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.316533 3662 catalog_manager.cc:5526] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 4 to 5, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.320916 7533 ts_tablet_manager.cc:927] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.321060 7533 tablet_copy_client.cc:323] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.321255 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:50395)
I20250112 21:06:45.321307 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:50395: session id = 67a5a7038ced40ddb6f45fdbfd99825c-8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:45.321938 3826 tablet_copy_source_session.cc:215] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.322458 7533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8880a7467c6a44da96f98c3cc368c9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.323400 7794 ts_tablet_manager.cc:927] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:45.323531 7794 tablet_copy_client.cc:323] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:45.323645 7533 tablet_copy_client.cc:806] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.323758 7533 tablet_copy_client.cc:670] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.324839 7533 tablet_copy_client.cc:538] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.325448 7533 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.328536 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 7493a3e50b0e4fce955005b60a713348 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:37341)
I20250112 21:06:45.328599 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 7493a3e50b0e4fce955005b60a713348 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:37341: session id = 67a5a7038ced40ddb6f45fdbfd99825c-7493a3e50b0e4fce955005b60a713348
I20250112 21:06:45.329305 4089 tablet_copy_source_session.cc:215] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.330025 7533 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.330134 7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7493a3e50b0e4fce955005b60a713348. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.330230 7533 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.330328 7533 ts_tablet_manager.cc:1397] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250112 21:06:45.330462 7533 raft_consensus.cc:357] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.330530 7533 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.330850 7533 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.331017 7533 ts_tablet_manager.cc:1428] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:45.331183 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-8880a7467c6a44da96f98c3cc368c9f0 received from {username='slave'} at 127.26.82.70:50395
I20250112 21:06:45.331234 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-8880a7467c6a44da96f98c3cc368c9f0 on tablet 8880a7467c6a44da96f98c3cc368c9f0 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.331358 7794 tablet_copy_client.cc:806] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.331457 7794 tablet_copy_client.cc:670] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.333570 7794 tablet_copy_client.cc:538] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.334214 7794 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.343046 7794 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.349483 7794 tablet_bootstrap.cc:492] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.349619 7794 ts_tablet_manager.cc:1397] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20250112 21:06:45.349776 7794 raft_consensus.cc:357] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.349871 7794 raft_consensus.cc:738] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.349953 7794 consensus_queue.cc:260] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.350116 7794 ts_tablet_manager.cc:1428] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:45.350311 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-7493a3e50b0e4fce955005b60a713348 received from {username='slave'} at 127.26.82.70:37341
I20250112 21:06:45.350370 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-7493a3e50b0e4fce955005b60a713348 on tablet 7493a3e50b0e4fce955005b60a713348 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.353556 7788 ts_tablet_manager.cc:927] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.353669 7788 tablet_copy_client.cc:323] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.353879 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:52565)
I20250112 21:06:45.353932 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:52565: session id = 38c8e8afd3f5411490f56ebf57aee109-8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:45.354471 3826 tablet_copy_source_session.cc:215] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.354976 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8880a7467c6a44da96f98c3cc368c9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.355909 7798 ts_tablet_manager.cc:927] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.356072 7798 tablet_copy_client.cc:323] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.356336 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet e82a54be68bc4c3ca23a516694e980eb from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49463)
I20250112 21:06:45.356393 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet e82a54be68bc4c3ca23a516694e980eb from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49463: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-e82a54be68bc4c3ca23a516694e980eb
I20250112 21:06:45.356448 7788 tablet_copy_client.cc:806] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.356561 7788 tablet_copy_client.cc:670] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.357052 3958 tablet_copy_source_session.cc:215] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.357443 7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82a54be68bc4c3ca23a516694e980eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.358887 7798 tablet_copy_client.cc:806] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.359006 7798 tablet_copy_client.cc:670] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.360869 7788 tablet_copy_client.cc:538] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.361593 7788 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.364605 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.369879 7788 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.370123 7788 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.370239 7788 ts_tablet_manager.cc:1397] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20250112 21:06:45.370381 7788 raft_consensus.cc:357] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.370819 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.371381 7788 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.371461 7788 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.371618 7788 ts_tablet_manager.cc:1428] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:45.371798 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-8880a7467c6a44da96f98c3cc368c9f0 received from {username='slave'} at 127.26.82.72:52565
I20250112 21:06:45.371845 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-8880a7467c6a44da96f98c3cc368c9f0 on tablet 8880a7467c6a44da96f98c3cc368c9f0 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.373274 7798 tablet_copy_client.cc:538] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.373940 7798 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.374894 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.381199 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.382025 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.383375 7506 ts_tablet_manager.cc:927] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.383512 7506 tablet_copy_client.cc:323] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.383745 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 74ff25b3286e45af8ddd38965062fca5 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49797)
I20250112 21:06:45.383798 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 74ff25b3286e45af8ddd38965062fca5 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49797: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-74ff25b3286e45af8ddd38965062fca5
I20250112 21:06:45.384253 4220 tablet_copy_source_session.cc:215] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.384505 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.384588 7506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ff25b3286e45af8ddd38965062fca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.384660 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.385684 7506 tablet_copy_client.cc:806] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.385784 7506 tablet_copy_client.cc:670] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.388705 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.394906 7798 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.395098 7798 tablet_bootstrap.cc:492] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.395188 7798 ts_tablet_manager.cc:1397] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20250112 21:06:45.395316 7798 raft_consensus.cc:357] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.395371 7798 raft_consensus.cc:738] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.395408 7798 consensus_queue.cc:260] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.395505 7798 ts_tablet_manager.cc:1428] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:45.395728 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-e82a54be68bc4c3ca23a516694e980eb received from {username='slave'} at 127.26.82.71:49463
I20250112 21:06:45.395782 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-e82a54be68bc4c3ca23a516694e980eb on tablet e82a54be68bc4c3ca23a516694e980eb with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.396728 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.397043 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.397085 7506 tablet_copy_client.cc:538] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.397707 7506 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.400148 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.401779 7506 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.402025 7506 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.402108 7506 ts_tablet_manager.cc:1397] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20250112 21:06:45.402220 7506 raft_consensus.cc:357] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.402279 7506 raft_consensus.cc:738] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.402369 7506 consensus_queue.cc:260] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.402865 7506 ts_tablet_manager.cc:1428] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:45.403326 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-74ff25b3286e45af8ddd38965062fca5 received from {username='slave'} at 127.26.82.71:49797
I20250112 21:06:45.403373 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-74ff25b3286e45af8ddd38965062fca5 on tablet 74ff25b3286e45af8ddd38965062fca5 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.406776 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.407701 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.407770 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.415685 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.415819 7506 ts_tablet_manager.cc:927] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.415936 7506 tablet_copy_client.cc:323] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.416280 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet c93763a09f7d4460b1874bf328fb06f5 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49463)
I20250112 21:06:45.416329 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet c93763a09f7d4460b1874bf328fb06f5 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49463: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-c93763a09f7d4460b1874bf328fb06f5
I20250112 21:06:45.416682 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.416853 3958 tablet_copy_source_session.cc:215] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.417685 7506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c93763a09f7d4460b1874bf328fb06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.419283 7506 tablet_copy_client.cc:806] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.419464 7506 tablet_copy_client.cc:670] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.422825 7788 ts_tablet_manager.cc:927] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.422963 7788 tablet_copy_client.cc:323] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.424121 7553 ts_tablet_manager.cc:927] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.424271 7553 tablet_copy_client.cc:323] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.424556 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 90f8eabb6a0d4292953e3dec2efec97d from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:53179)
I20250112 21:06:45.424605 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 90f8eabb6a0d4292953e3dec2efec97d from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:53179: session id = 6f7f47d877a8407185013dba2959fddd-90f8eabb6a0d4292953e3dec2efec97d
I20250112 21:06:45.425055 3958 tablet_copy_source_session.cc:215] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.425462 7553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f8eabb6a0d4292953e3dec2efec97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.426469 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:33877)
I20250112 21:06:45.426525 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:33877: session id = 38c8e8afd3f5411490f56ebf57aee109-de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:45.426810 7553 tablet_copy_client.cc:806] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.426914 7553 tablet_copy_client.cc:670] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.427043 4220 tablet_copy_source_session.cc:215] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.427373 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.428519 7788 tablet_copy_client.cc:806] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.428625 7788 tablet_copy_client.cc:670] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.433768 7506 tablet_copy_client.cc:538] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.434437 5733 raft_consensus.cc:1212] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.434691 7506 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.436933 7553 tablet_copy_client.cc:538] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.437672 7553 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.437680 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.443428 7506 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.443732 7506 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.443908 7506 ts_tablet_manager.cc:1397] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20250112 21:06:45.444077 7506 raft_consensus.cc:357] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.444202 7506 raft_consensus.cc:738] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.444299 7506 consensus_queue.cc:260] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.444530 7506 ts_tablet_manager.cc:1428] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:45.445791 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-c93763a09f7d4460b1874bf328fb06f5 received from {username='slave'} at 127.26.82.71:49463
I20250112 21:06:45.445845 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-c93763a09f7d4460b1874bf328fb06f5 on tablet c93763a09f7d4460b1874bf328fb06f5 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.446305 7553 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.446493 7553 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.446591 7553 ts_tablet_manager.cc:1397] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20250112 21:06:45.446790 7553 raft_consensus.cc:357] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.447073 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.446943 7553 raft_consensus.cc:738] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.447209 7553 consensus_queue.cc:260] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.447579 7553 ts_tablet_manager.cc:1428] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:45.447803 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-90f8eabb6a0d4292953e3dec2efec97d received from {username='slave'} at 127.26.82.69:53179
I20250112 21:06:45.447854 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-90f8eabb6a0d4292953e3dec2efec97d on tablet 90f8eabb6a0d4292953e3dec2efec97d with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.449123 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.451195 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.451416 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.451977 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.454136 7553 ts_tablet_manager.cc:927] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.454244 7553 tablet_copy_client.cc:323] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.456555 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.457396 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 917b9e7ed0ed43efa0a60ff461e4f213 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:41067)
I20250112 21:06:45.457474 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 917b9e7ed0ed43efa0a60ff461e4f213 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:41067: session id = 6f7f47d877a8407185013dba2959fddd-917b9e7ed0ed43efa0a60ff461e4f213
I20250112 21:06:45.457599 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.458024 3826 tablet_copy_source_session.cc:215] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.459091 7553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917b9e7ed0ed43efa0a60ff461e4f213. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.460218 7553 tablet_copy_client.cc:806] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.460336 7553 tablet_copy_client.cc:670] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.460551 7788 tablet_copy_client.cc:538] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.461032 7553 tablet_copy_client.cc:538] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.461397 7788 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.461695 7553 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.465358 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.468873 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.469282 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.469982 7506 ts_tablet_manager.cc:927] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.470108 7506 tablet_copy_client.cc:323] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.470306 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 90f8eabb6a0d4292953e3dec2efec97d from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49463)
I20250112 21:06:45.470362 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 90f8eabb6a0d4292953e3dec2efec97d from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49463: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-90f8eabb6a0d4292953e3dec2efec97d
I20250112 21:06:45.470824 3958 tablet_copy_source_session.cc:215] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.471128 7506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f8eabb6a0d4292953e3dec2efec97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.472100 7788 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.472306 7788 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.472399 7788 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20250112 21:06:45.472463 7506 tablet_copy_client.cc:806] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.472525 7788 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.472582 7506 tablet_copy_client.cc:670] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.472608 7788 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.472674 7788 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.472783 7788 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.004s sys 0.000s
I20250112 21:06:45.472975 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-de6b6ac6eeff431b9ac8cce47ea3a59e received from {username='slave'} at 127.26.82.72:33877
I20250112 21:06:45.473032 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-de6b6ac6eeff431b9ac8cce47ea3a59e on tablet de6b6ac6eeff431b9ac8cce47ea3a59e with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.473101 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.474938 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.477205 7553 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.477406 7553 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.477524 7553 ts_tablet_manager.cc:1397] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.001s
I20250112 21:06:45.485183 7553 raft_consensus.cc:357] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.485270 7553 raft_consensus.cc:738] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.485327 7553 consensus_queue.cc:260] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.485610 7553 ts_tablet_manager.cc:1428] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.008s user 0.000s sys 0.000s
I20250112 21:06:45.485785 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-917b9e7ed0ed43efa0a60ff461e4f213 received from {username='slave'} at 127.26.82.69:41067
I20250112 21:06:45.485837 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-917b9e7ed0ed43efa0a60ff461e4f213 on tablet 917b9e7ed0ed43efa0a60ff461e4f213 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.486013 5470 raft_consensus.cc:1212] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.488333 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.491698 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.492789 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.495815 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.496593 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.502513 7798 ts_tablet_manager.cc:927] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.502677 7798 tablet_copy_client.cc:323] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.503298 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.507817 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 917b9e7ed0ed43efa0a60ff461e4f213 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:34353)
I20250112 21:06:45.507874 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 917b9e7ed0ed43efa0a60ff461e4f213 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:34353: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-917b9e7ed0ed43efa0a60ff461e4f213
I20250112 21:06:45.508414 3826 tablet_copy_source_session.cc:215] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.508843 7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917b9e7ed0ed43efa0a60ff461e4f213. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.510121 7798 tablet_copy_client.cc:806] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.510247 7798 tablet_copy_client.cc:670] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.510329 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.510481 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.510849 7798 tablet_copy_client.cc:538] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.511356 7798 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.515327 7798 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.515599 7798 tablet_bootstrap.cc:492] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.515761 7798 ts_tablet_manager.cc:1397] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.003s
I20250112 21:06:45.515940 7798 raft_consensus.cc:357] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.516099 7798 raft_consensus.cc:738] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.516258 7798 consensus_queue.cc:260] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.516624 7798 ts_tablet_manager.cc:1428] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:45.516945 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-917b9e7ed0ed43efa0a60ff461e4f213 received from {username='slave'} at 127.26.82.71:34353
I20250112 21:06:45.517004 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-917b9e7ed0ed43efa0a60ff461e4f213 on tablet 917b9e7ed0ed43efa0a60ff461e4f213 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.520898 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.524277 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.525730 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.527966 7506 tablet_copy_client.cc:538] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.528784 7506 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.533038 7794 ts_tablet_manager.cc:927] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.533210 7794 tablet_copy_client.cc:323] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.533409 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 1fb3deb52b8c4cceb1814858ea30e94f from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:48269)
I20250112 21:06:45.533463 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 1fb3deb52b8c4cceb1814858ea30e94f from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:48269: session id = 67a5a7038ced40ddb6f45fdbfd99825c-1fb3deb52b8c4cceb1814858ea30e94f
I20250112 21:06:45.533994 4220 tablet_copy_source_session.cc:215] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.534322 7506 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.534428 7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb3deb52b8c4cceb1814858ea30e94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.534530 7506 tablet_bootstrap.cc:492] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.534621 7506 ts_tablet_manager.cc:1397] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20250112 21:06:45.534745 7506 raft_consensus.cc:357] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.534806 7506 raft_consensus.cc:738] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.534848 7506 consensus_queue.cc:260] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.534930 7506 ts_tablet_manager.cc:1428] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.535204 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-90f8eabb6a0d4292953e3dec2efec97d received from {username='slave'} at 127.26.82.71:49463
I20250112 21:06:45.535252 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-90f8eabb6a0d4292953e3dec2efec97d on tablet 90f8eabb6a0d4292953e3dec2efec97d with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.535719 7794 tablet_copy_client.cc:806] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.535818 7794 tablet_copy_client.cc:670] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.542280 7794 tablet_copy_client.cc:538] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.543049 7794 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.548906 7794 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.549110 7794 tablet_bootstrap.cc:492] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.549211 7794 ts_tablet_manager.cc:1397] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250112 21:06:45.549333 7794 raft_consensus.cc:357] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.549403 7794 raft_consensus.cc:738] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.549919 7794 consensus_queue.cc:260] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.550278 7794 ts_tablet_manager.cc:1428] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:45.550546 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-1fb3deb52b8c4cceb1814858ea30e94f received from {username='slave'} at 127.26.82.70:48269
I20250112 21:06:45.550606 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-1fb3deb52b8c4cceb1814858ea30e94f on tablet 1fb3deb52b8c4cceb1814858ea30e94f with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.556114 5602 raft_consensus.cc:1212] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.571307 7553 ts_tablet_manager.cc:927] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.571475 7553 tablet_copy_client.cc:323] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.572250 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 56e0dcaf59aa4303aff6810c70092184 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:41067)
I20250112 21:06:45.572307 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 56e0dcaf59aa4303aff6810c70092184 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:41067: session id = 6f7f47d877a8407185013dba2959fddd-56e0dcaf59aa4303aff6810c70092184
I20250112 21:06:45.573175 3826 tablet_copy_source_session.cc:215] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.573861 7553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e0dcaf59aa4303aff6810c70092184. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.574008 8075 ts_tablet_manager.cc:927] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.574167 8075 tablet_copy_client.cc:323] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.574463 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:47595)
I20250112 21:06:45.574539 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:47595: session id = 6f7f47d877a8407185013dba2959fddd-de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:45.575093 4220 tablet_copy_source_session.cc:215] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.575121 7553 tablet_copy_client.cc:806] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.575233 7553 tablet_copy_client.cc:670] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.575529 8075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.576851 8075 tablet_copy_client.cc:806] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.576954 8075 tablet_copy_client.cc:670] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.577574 8075 tablet_copy_client.cc:538] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.578270 8075 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.578873 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.579726 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.580106 8220 ts_tablet_manager.cc:927] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.580416 8220 tablet_copy_client.cc:323] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.580873 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet b6ba48c8dc7e4ee485f85ee94aca256b from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:47595)
I20250112 21:06:45.580924 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet b6ba48c8dc7e4ee485f85ee94aca256b from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:47595: session id = 6f7f47d877a8407185013dba2959fddd-b6ba48c8dc7e4ee485f85ee94aca256b
I20250112 21:06:45.581382 4220 tablet_copy_source_session.cc:215] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.581719 8220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ba48c8dc7e4ee485f85ee94aca256b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.582935 8220 tablet_copy_client.cc:806] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.583195 8075 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.583230 8220 tablet_copy_client.cc:670] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.583391 8075 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.583479 8075 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20250112 21:06:45.583628 7788 ts_tablet_manager.cc:927] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.583631 8075 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.583703 8075 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.583770 7788 tablet_copy_client.cc:323] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.583753 8075 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.583859 8075 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.584017 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-de6b6ac6eeff431b9ac8cce47ea3a59e received from {username='slave'} at 127.26.82.69:47595
I20250112 21:06:45.584040 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 56e0dcaf59aa4303aff6810c70092184 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:52565)
I20250112 21:06:45.584066 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-de6b6ac6eeff431b9ac8cce47ea3a59e on tablet de6b6ac6eeff431b9ac8cce47ea3a59e with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.584085 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 56e0dcaf59aa4303aff6810c70092184 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:52565: session id = 38c8e8afd3f5411490f56ebf57aee109-56e0dcaf59aa4303aff6810c70092184
I20250112 21:06:45.584543 3826 tablet_copy_source_session.cc:215] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.585135 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e0dcaf59aa4303aff6810c70092184. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.586130 7788 tablet_copy_client.cc:806] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.586236 7788 tablet_copy_client.cc:670] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.588337 7788 tablet_copy_client.cc:538] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.589092 7788 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.592446 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.593348 7788 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.593523 7788 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.593616 7788 ts_tablet_manager.cc:1397] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250112 21:06:45.593726 7788 raft_consensus.cc:357] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.593782 7788 raft_consensus.cc:738] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.593830 7788 consensus_queue.cc:260] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.594126 7506 ts_tablet_manager.cc:927] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.594247 7506 tablet_copy_client.cc:323] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.594415 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet b3b6276ae83948b6b4ea2771d9ce8355 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:34353)
I20250112 21:06:45.594463 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet b3b6276ae83948b6b4ea2771d9ce8355 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:34353: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-b3b6276ae83948b6b4ea2771d9ce8355
I20250112 21:06:45.594465 7788 ts_tablet_manager.cc:1428] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:45.594872 3826 tablet_copy_source_session.cc:215] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.595106 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-56e0dcaf59aa4303aff6810c70092184 received from {username='slave'} at 127.26.82.72:52565
I20250112 21:06:45.595150 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-56e0dcaf59aa4303aff6810c70092184 on tablet 56e0dcaf59aa4303aff6810c70092184 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.595204 7506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b6276ae83948b6b4ea2771d9ce8355. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.596081 7506 tablet_copy_client.cc:806] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.596215 7506 tablet_copy_client.cc:670] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.597397 7506 tablet_copy_client.cc:538] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.598078 7506 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.600943 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.601192 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.601905 7506 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.602129 7506 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.602213 7506 ts_tablet_manager.cc:1397] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20250112 21:06:45.602322 7506 raft_consensus.cc:357] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.602396 7506 raft_consensus.cc:738] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.602614 7506 consensus_queue.cc:260] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.603118 7506 ts_tablet_manager.cc:1428] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:45.603304 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-b3b6276ae83948b6b4ea2771d9ce8355 received from {username='slave'} at 127.26.82.71:34353
I20250112 21:06:45.603356 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-b3b6276ae83948b6b4ea2771d9ce8355 on tablet b3b6276ae83948b6b4ea2771d9ce8355 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.607658 7794 ts_tablet_manager.cc:927] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.607789 7794 tablet_copy_client.cc:323] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.607949 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet b3b6276ae83948b6b4ea2771d9ce8355 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:50395)
I20250112 21:06:45.607986 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet b3b6276ae83948b6b4ea2771d9ce8355 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:50395: session id = 67a5a7038ced40ddb6f45fdbfd99825c-b3b6276ae83948b6b4ea2771d9ce8355
I20250112 21:06:45.608355 3826 tablet_copy_source_session.cc:215] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.608693 7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b6276ae83948b6b4ea2771d9ce8355. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.610766 7794 tablet_copy_client.cc:806] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.610877 7794 tablet_copy_client.cc:670] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.610989 8075 ts_tablet_manager.cc:927] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.611114 8075 tablet_copy_client.cc:323] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.611368 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet c93763a09f7d4460b1874bf328fb06f5 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:53179)
I20250112 21:06:45.611441 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet c93763a09f7d4460b1874bf328fb06f5 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:53179: session id = 6f7f47d877a8407185013dba2959fddd-c93763a09f7d4460b1874bf328fb06f5
I20250112 21:06:45.611963 3958 tablet_copy_source_session.cc:215] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.612237 8075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c93763a09f7d4460b1874bf328fb06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.612694 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.613234 7506 ts_tablet_manager.cc:927] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.613335 8075 tablet_copy_client.cc:806] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.613370 7506 tablet_copy_client.cc:323] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.613443 8075 tablet_copy_client.cc:670] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.613819 3826 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 345f4bf45767475ca481525f3d26a34a from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:34353)
I20250112 21:06:45.613876 3826 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 345f4bf45767475ca481525f3d26a34a from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:34353: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-345f4bf45767475ca481525f3d26a34a
I20250112 21:06:45.614282 3826 tablet_copy_source_session.cc:215] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.614588 7506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 345f4bf45767475ca481525f3d26a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.615680 7506 tablet_copy_client.cc:806] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.615774 7506 tablet_copy_client.cc:670] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.616978 7788 ts_tablet_manager.cc:927] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.617983 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.619634 7788 tablet_copy_client.cc:323] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.619866 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet b6ba48c8dc7e4ee485f85ee94aca256b from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:33877)
I20250112 21:06:45.619916 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet b6ba48c8dc7e4ee485f85ee94aca256b from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:33877: session id = 38c8e8afd3f5411490f56ebf57aee109-b6ba48c8dc7e4ee485f85ee94aca256b
I20250112 21:06:45.620523 4220 tablet_copy_source_session.cc:215] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.620934 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ba48c8dc7e4ee485f85ee94aca256b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.621994 7506 tablet_copy_client.cc:538] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.622085 8075 tablet_copy_client.cc:538] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.622254 7788 tablet_copy_client.cc:806] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.622352 7788 tablet_copy_client.cc:670] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.622707 7506 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.622726 8075 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.624679 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.626113 7788 tablet_copy_client.cc:538] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.626871 7788 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.627014 8075 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.627210 8075 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.627300 8075 ts_tablet_manager.cc:1397] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250112 21:06:45.627403 8075 raft_consensus.cc:357] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.627467 8075 raft_consensus.cc:738] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.627559 8075 consensus_queue.cc:260] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.627938 8075 ts_tablet_manager.cc:1428] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:45.628122 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-c93763a09f7d4460b1874bf328fb06f5 received from {username='slave'} at 127.26.82.69:53179
I20250112 21:06:45.628168 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-c93763a09f7d4460b1874bf328fb06f5 on tablet c93763a09f7d4460b1874bf328fb06f5 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.631995 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.635989 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.636897 7788 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.636889 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.637107 7788 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.637204 7788 ts_tablet_manager.cc:1397] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20250112 21:06:45.637316 7788 raft_consensus.cc:357] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.637389 7788 raft_consensus.cc:738] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.637456 7788 consensus_queue.cc:260] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.637791 7788 ts_tablet_manager.cc:1428] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:45.637960 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-b6ba48c8dc7e4ee485f85ee94aca256b received from {username='slave'} at 127.26.82.72:33877
I20250112 21:06:45.638020 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-b6ba48c8dc7e4ee485f85ee94aca256b on tablet b6ba48c8dc7e4ee485f85ee94aca256b with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.639039 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.642223 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.648933 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.651402 8220 tablet_copy_client.cc:538] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.652257 8220 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.653656 7553 tablet_copy_client.cc:538] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.654352 7553 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.655496 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.658882 7794 tablet_copy_client.cc:538] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.659657 7794 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.662472 7553 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.662653 7553 tablet_bootstrap.cc:492] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.662756 7553 ts_tablet_manager.cc:1397] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20250112 21:06:45.662868 7553 raft_consensus.cc:357] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.662940 7553 raft_consensus.cc:738] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.663388 5864 raft_consensus.cc:1212] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.664372 8220 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.664892 8220 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.665004 8220 ts_tablet_manager.cc:1397] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20250112 21:06:45.665099 8220 raft_consensus.cc:357] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.665169 8220 raft_consensus.cc:738] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.665217 8220 consensus_queue.cc:260] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.665861 5843 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.667145 7506 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.667358 7506 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.667459 7506 ts_tablet_manager.cc:1397] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.001s
I20250112 21:06:45.667573 7506 raft_consensus.cc:357] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.667634 7506 raft_consensus.cc:738] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.668025 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.668571 7506 consensus_queue.cc:260] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.668838 7506 ts_tablet_manager.cc:1428] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:45.669173 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-345f4bf45767475ca481525f3d26a34a received from {username='slave'} at 127.26.82.71:34353
I20250112 21:06:45.669225 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-345f4bf45767475ca481525f3d26a34a on tablet 345f4bf45767475ca481525f3d26a34a with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.669356 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.672701 7553 consensus_queue.cc:260] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.672890 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.673274 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.679459 7798 ts_tablet_manager.cc:927] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:45.679598 7798 tablet_copy_client.cc:323] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:45.680181 7788 ts_tablet_manager.cc:927] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.680338 7788 tablet_copy_client.cc:323] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.680898 7553 ts_tablet_manager.cc:1428] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.018s user 0.000s sys 0.001s
I20250112 21:06:45.680758 8220 ts_tablet_manager.cc:1428] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.016s user 0.001s sys 0.000s
I20250112 21:06:45.681026 7794 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.681228 7794 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.681314 7794 ts_tablet_manager.cc:1397] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.004s
I20250112 21:06:45.681421 7794 raft_consensus.cc:357] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.681492 7794 raft_consensus.cc:738] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.681671 7794 consensus_queue.cc:260] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.682085 7794 ts_tablet_manager.cc:1428] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:45.682250 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.682545 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 5da4fd48e1f24680babd4f1874b31b5f from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:33877)
I20250112 21:06:45.682605 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 5da4fd48e1f24680babd4f1874b31b5f from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:33877: session id = 38c8e8afd3f5411490f56ebf57aee109-5da4fd48e1f24680babd4f1874b31b5f
I20250112 21:06:45.683259 4220 tablet_copy_source_session.cc:215] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.683944 8075 ts_tablet_manager.cc:927] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:45.684087 8075 tablet_copy_client.cc:323] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:45.684157 3826 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-b3b6276ae83948b6b4ea2771d9ce8355 received from {username='slave'} at 127.26.82.70:50395
I20250112 21:06:45.684199 3826 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-b3b6276ae83948b6b4ea2771d9ce8355 on tablet b3b6276ae83948b6b4ea2771d9ce8355 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.684319 3827 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-56e0dcaf59aa4303aff6810c70092184 received from {username='slave'} at 127.26.82.69:41067
I20250112 21:06:45.684357 3827 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-56e0dcaf59aa4303aff6810c70092184 on tablet 56e0dcaf59aa4303aff6810c70092184 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.685808 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5da4fd48e1f24680babd4f1874b31b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.686144 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.687022 7788 tablet_copy_client.cc:806] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.687137 7788 tablet_copy_client.cc:670] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.690603 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-b6ba48c8dc7e4ee485f85ee94aca256b received from {username='slave'} at 127.26.82.69:47595
I20250112 21:06:45.690654 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-b6ba48c8dc7e4ee485f85ee94aca256b on tablet b6ba48c8dc7e4ee485f85ee94aca256b with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.691943 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 7249078cad3e4bc3ab2b62d8887987df from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:60217)
I20250112 21:06:45.692003 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 7249078cad3e4bc3ab2b62d8887987df from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:60217: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:45.692487 4089 tablet_copy_source_session.cc:215] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.692909 7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7249078cad3e4bc3ab2b62d8887987df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.693020 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 0bad642a13544a3caa0ca9bcaa4a1257 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:35291)
I20250112 21:06:45.693063 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 0bad642a13544a3caa0ca9bcaa4a1257 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:35291: session id = 6f7f47d877a8407185013dba2959fddd-0bad642a13544a3caa0ca9bcaa4a1257
I20250112 21:06:45.693593 4089 tablet_copy_source_session.cc:215] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.693857 8075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad642a13544a3caa0ca9bcaa4a1257. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.694236 7798 tablet_copy_client.cc:806] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.694339 7798 tablet_copy_client.cc:670] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.695107 8075 tablet_copy_client.cc:806] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.695214 8075 tablet_copy_client.cc:670] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.695392 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.695422 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.695572 7794 ts_tablet_manager.cc:927] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.695693 7794 tablet_copy_client.cc:323] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.697223 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 7fc2e530f44745948c4607c5f81b1d0e from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:48269)
I20250112 21:06:45.697258 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.697273 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 7fc2e530f44745948c4607c5f81b1d0e from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:48269: session id = 67a5a7038ced40ddb6f45fdbfd99825c-7fc2e530f44745948c4607c5f81b1d0e
I20250112 21:06:45.697789 4220 tablet_copy_source_session.cc:215] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.698918 7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc2e530f44745948c4607c5f81b1d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.700099 7794 tablet_copy_client.cc:806] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.700192 7794 tablet_copy_client.cc:670] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.703060 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.703678 5733 raft_consensus.cc:1212] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.705451 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.710852 7794 tablet_copy_client.cc:538] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.710860 7788 tablet_copy_client.cc:538] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.711580 7794 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.711679 7788 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.713660 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.714080 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.716862 8075 tablet_copy_client.cc:538] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.717626 8075 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.719903 5602 raft_consensus.cc:1212] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.720695 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.721393 7798 tablet_copy_client.cc:538] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.722169 7798 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.724110 8220 ts_tablet_manager.cc:927] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:45.724215 8220 tablet_copy_client.cc:323] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:45.724385 3827 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 345f4bf45767475ca481525f3d26a34a from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:41067)
I20250112 21:06:45.724431 3827 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 345f4bf45767475ca481525f3d26a34a from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:41067: session id = 6f7f47d877a8407185013dba2959fddd-345f4bf45767475ca481525f3d26a34a
I20250112 21:06:45.724985 3827 tablet_copy_source_session.cc:215] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.725272 8220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 345f4bf45767475ca481525f3d26a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.726213 7798 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.726397 7798 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.726481 7798 ts_tablet_manager.cc:1397] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20250112 21:06:45.726616 8220 tablet_copy_client.cc:806] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.726593 7798 raft_consensus.cc:357] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.726677 7798 raft_consensus.cc:738] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.726729 8220 tablet_copy_client.cc:670] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.726807 7794 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.726780 7798 consensus_queue.cc:260] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.727085 7798 ts_tablet_manager.cc:1428] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250112 21:06:45.727501 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-7249078cad3e4bc3ab2b62d8887987df received from {username='slave'} at 127.26.82.71:60217
I20250112 21:06:45.727553 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-7249078cad3e4bc3ab2b62d8887987df on tablet 7249078cad3e4bc3ab2b62d8887987df with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.729650 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.732419 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.733132 7594 ts_tablet_manager.cc:927] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:45.733285 7594 tablet_copy_client.cc:323] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:45.733525 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 0bad642a13544a3caa0ca9bcaa4a1257 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:54097)
I20250112 21:06:45.733587 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 0bad642a13544a3caa0ca9bcaa4a1257 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:54097: session id = 38c8e8afd3f5411490f56ebf57aee109-0bad642a13544a3caa0ca9bcaa4a1257
I20250112 21:06:45.734196 4089 tablet_copy_source_session.cc:215] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.734576 7594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bad642a13544a3caa0ca9bcaa4a1257. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.735743 7594 tablet_copy_client.cc:806] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.735860 7594 tablet_copy_client.cc:670] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.736056 8075 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.738267 7594 tablet_copy_client.cc:538] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.738864 7788 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.738936 7594 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.739063 7788 tablet_bootstrap.cc:492] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.739153 7788 ts_tablet_manager.cc:1397] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20250112 21:06:45.739264 7788 raft_consensus.cc:357] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.739337 7788 raft_consensus.cc:738] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.739391 7788 consensus_queue.cc:260] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.739564 7788 ts_tablet_manager.cc:1428] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.002s sys 0.000s
I20250112 21:06:45.741046 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-5da4fd48e1f24680babd4f1874b31b5f received from {username='slave'} at 127.26.82.72:33877
I20250112 21:06:45.741107 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-5da4fd48e1f24680babd4f1874b31b5f on tablet 5da4fd48e1f24680babd4f1874b31b5f with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.742055 7788 ts_tablet_manager.cc:927] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:45.742173 7788 tablet_copy_client.cc:323] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:45.742318 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 7249078cad3e4bc3ab2b62d8887987df from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:54097)
I20250112 21:06:45.742359 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 7249078cad3e4bc3ab2b62d8887987df from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:54097: session id = 38c8e8afd3f5411490f56ebf57aee109-7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:45.742882 4089 tablet_copy_source_session.cc:215] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.743211 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7249078cad3e4bc3ab2b62d8887987df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.744436 7788 tablet_copy_client.cc:806] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.744561 7788 tablet_copy_client.cc:670] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.744887 7794 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.745025 7794 ts_tablet_manager.cc:1397] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.001s
I20250112 21:06:45.745184 7794 raft_consensus.cc:357] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.745271 7794 raft_consensus.cc:738] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.745322 7794 consensus_queue.cc:260] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.745422 7794 ts_tablet_manager.cc:1428] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.746376 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-7fc2e530f44745948c4607c5f81b1d0e received from {username='slave'} at 127.26.82.70:48269
I20250112 21:06:45.746426 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-7fc2e530f44745948c4607c5f81b1d0e on tablet 7fc2e530f44745948c4607c5f81b1d0e with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.746467 7788 tablet_copy_client.cc:538] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.747125 7788 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.748615 7594 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.749531 7594 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.749629 7594 ts_tablet_manager.cc:1397] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.002s
I20250112 21:06:45.749740 7594 raft_consensus.cc:357] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.749799 7594 raft_consensus.cc:738] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.749843 7594 consensus_queue.cc:260] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.749967 7594 ts_tablet_manager.cc:1428] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:45.750139 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-0bad642a13544a3caa0ca9bcaa4a1257 received from {username='slave'} at 127.26.82.72:54097
I20250112 21:06:45.750181 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-0bad642a13544a3caa0ca9bcaa4a1257 on tablet 0bad642a13544a3caa0ca9bcaa4a1257 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.752833 8075 tablet_bootstrap.cc:492] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.752952 8075 ts_tablet_manager.cc:1397] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.003s
I20250112 21:06:45.753085 8075 raft_consensus.cc:357] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.753170 8075 raft_consensus.cc:738] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.753222 8075 consensus_queue.cc:260] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.753319 8075 ts_tablet_manager.cc:1428] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.754148 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-0bad642a13544a3caa0ca9bcaa4a1257 received from {username='slave'} at 127.26.82.69:35291
I20250112 21:06:45.754199 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-0bad642a13544a3caa0ca9bcaa4a1257 on tablet 0bad642a13544a3caa0ca9bcaa4a1257 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.755107 8220 tablet_copy_client.cc:538] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.755867 8220 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.757802 7788 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.761457 8220 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.761797 8220 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.761921 8220 ts_tablet_manager.cc:1397] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20250112 21:06:45.762053 8220 raft_consensus.cc:357] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.762125 8220 raft_consensus.cc:738] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.762199 8220 consensus_queue.cc:260] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:45.762415 8220 ts_tablet_manager.cc:1428] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:45.762670 3827 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-345f4bf45767475ca481525f3d26a34a received from {username='slave'} at 127.26.82.69:41067
I20250112 21:06:45.762724 3827 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-345f4bf45767475ca481525f3d26a34a on tablet 345f4bf45767475ca481525f3d26a34a with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.763202 8075 ts_tablet_manager.cc:927] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.763319 8075 tablet_copy_client.cc:323] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.763588 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 0fce4710d0784d02991a6465522323d1 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:53179)
I20250112 21:06:45.763649 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 0fce4710d0784d02991a6465522323d1 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:53179: session id = 6f7f47d877a8407185013dba2959fddd-0fce4710d0784d02991a6465522323d1
I20250112 21:06:45.764153 3958 tablet_copy_source_session.cc:215] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.764636 8075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce4710d0784d02991a6465522323d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.765733 8075 tablet_copy_client.cc:806] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.765838 8075 tablet_copy_client.cc:670] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.767830 8075 tablet_copy_client.cc:538] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.768491 8075 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:45.774034 8075 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.774216 8075 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:45.774292 8075 ts_tablet_manager.cc:1397] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20250112 21:06:45.774390 8075 raft_consensus.cc:357] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.774446 8075 raft_consensus.cc:738] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.775007 8075 consensus_queue.cc:260] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.775264 8075 ts_tablet_manager.cc:1428] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20250112 21:06:45.775440 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-0fce4710d0784d02991a6465522323d1 received from {username='slave'} at 127.26.82.69:53179
I20250112 21:06:45.775491 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-0fce4710d0784d02991a6465522323d1 on tablet 0fce4710d0784d02991a6465522323d1 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:45.777433 7788 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.777550 7788 ts_tablet_manager.cc:1397] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20250112 21:06:45.777689 7788 raft_consensus.cc:357] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.777774 7788 raft_consensus.cc:738] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.777814 7788 consensus_queue.cc:260] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.777896 7788 ts_tablet_manager.cc:1428] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.778187 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-7249078cad3e4bc3ab2b62d8887987df received from {username='slave'} at 127.26.82.72:54097
I20250112 21:06:45.778242 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-7249078cad3e4bc3ab2b62d8887987df on tablet 7249078cad3e4bc3ab2b62d8887987df with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.783197 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.784430 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.787154 5602 raft_consensus.cc:1212] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.787966 7788 ts_tablet_manager.cc:927] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.788106 7788 tablet_copy_client.cc:323] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.788558 7794 ts_tablet_manager.cc:927] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.788853 7594 ts_tablet_manager.cc:927] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:45.788969 7594 tablet_copy_client.cc:323] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:45.789213 7794 tablet_copy_client.cc:323] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.789229 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 7fc2e530f44745948c4607c5f81b1d0e from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:33877)
I20250112 21:06:45.789278 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 7fc2e530f44745948c4607c5f81b1d0e from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:33877: session id = 38c8e8afd3f5411490f56ebf57aee109-7fc2e530f44745948c4607c5f81b1d0e
I20250112 21:06:45.789466 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet c16b940a9e564da89ec84b46a6cf42d7 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:38803)
I20250112 21:06:45.789530 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet c16b940a9e564da89ec84b46a6cf42d7 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:38803: session id = 67a5a7038ced40ddb6f45fdbfd99825c-c16b940a9e564da89ec84b46a6cf42d7
I20250112 21:06:45.789812 4220 tablet_copy_source_session.cc:215] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.790135 3958 tablet_copy_source_session.cc:215] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.790345 7594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc2e530f44745948c4607c5f81b1d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.791267 7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16b940a9e564da89ec84b46a6cf42d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.791644 7594 tablet_copy_client.cc:806] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.791745 7594 tablet_copy_client.cc:670] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.792660 7794 tablet_copy_client.cc:806] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.792797 7794 tablet_copy_client.cc:670] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.793346 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.794938 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 0fce4710d0784d02991a6465522323d1 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:58797)
I20250112 21:06:45.794997 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 0fce4710d0784d02991a6465522323d1 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:58797: session id = 38c8e8afd3f5411490f56ebf57aee109-0fce4710d0784d02991a6465522323d1
I20250112 21:06:45.795373 3958 tablet_copy_source_session.cc:215] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.796919 7788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fce4710d0784d02991a6465522323d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.798023 7788 tablet_copy_client.cc:806] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.798132 7788 tablet_copy_client.cc:670] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.799156 7788 tablet_copy_client.cc:538] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.799782 7788 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.800292 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.801414 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.802352 5733 raft_consensus.cc:1212] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.804138 7788 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.804337 7788 tablet_bootstrap.cc:492] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.804452 7788 ts_tablet_manager.cc:1397] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250112 21:06:45.804565 7788 raft_consensus.cc:357] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.804636 7788 raft_consensus.cc:738] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.805042 7788 consensus_queue.cc:260] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:45.805205 7788 ts_tablet_manager.cc:1428] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:45.805433 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-0fce4710d0784d02991a6465522323d1 received from {username='slave'} at 127.26.82.72:58797
I20250112 21:06:45.805480 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-0fce4710d0784d02991a6465522323d1 on tablet 0fce4710d0784d02991a6465522323d1 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.808209 7594 tablet_copy_client.cc:538] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.808816 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.809523 7594 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:45.812995 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.813056 5602 raft_consensus.cc:1212] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.814972 7594 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.815181 7594 tablet_bootstrap.cc:492] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:45.815276 7594 ts_tablet_manager.cc:1397] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250112 21:06:45.815394 7594 raft_consensus.cc:357] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.815459 7594 raft_consensus.cc:738] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.815639 7594 consensus_queue.cc:260] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:45.815990 7594 ts_tablet_manager.cc:1428] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20250112 21:06:45.816288 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-7fc2e530f44745948c4607c5f81b1d0e received from {username='slave'} at 127.26.82.72:33877
I20250112 21:06:45.816351 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-7fc2e530f44745948c4607c5f81b1d0e on tablet 7fc2e530f44745948c4607c5f81b1d0e with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:45.817414 7794 tablet_copy_client.cc:538] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.817478 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.818226 7794 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:45.825408 5864 raft_consensus.cc:1212] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.829605 5470 raft_consensus.cc:1212] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.837591 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.837607 5602 raft_consensus.cc:1212] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.838073 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.840451 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.843873 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.853958 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.855648 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.860177 5602 raft_consensus.cc:1212] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.860522 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.863130 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.864135 5733 raft_consensus.cc:1212] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.864955 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.868232 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.869611 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.873453 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.873505 5864 raft_consensus.cc:1212] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.877269 7798 ts_tablet_manager.cc:927] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:45.877382 7798 tablet_copy_client.cc:323] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:45.877599 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet c16b940a9e564da89ec84b46a6cf42d7 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49463)
I20250112 21:06:45.877647 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet c16b940a9e564da89ec84b46a6cf42d7 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49463: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-c16b940a9e564da89ec84b46a6cf42d7
I20250112 21:06:45.878098 3958 tablet_copy_source_session.cc:215] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:45.878621 7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16b940a9e564da89ec84b46a6cf42d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:45.882575 3936 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.882726 7798 tablet_copy_client.cc:806] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:45.882836 7798 tablet_copy_client.cc:670] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:45.883060 4067 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.883257 6708 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.883539 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.883885 5733 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:45.884176 6635 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:45.885735 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.886718 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.888357 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.890507 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.891263 5864 raft_consensus.cc:1212] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.899524 7794 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.899709 7794 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:45.899824 7794 ts_tablet_manager.cc:1397] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.003s
I20250112 21:06:45.899957 7794 raft_consensus.cc:357] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.900027 7794 raft_consensus.cc:738] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.900199 7794 consensus_queue.cc:260] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.900501 7794 ts_tablet_manager.cc:1428] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:45.901050 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-c16b940a9e564da89ec84b46a6cf42d7 received from {username='slave'} at 127.26.82.70:38803
I20250112 21:06:45.901103 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-c16b940a9e564da89ec84b46a6cf42d7 on tablet c16b940a9e564da89ec84b46a6cf42d7 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:45.901455 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.902598 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.903749 6708 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.904817 4067 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.906157 3660 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.906401 5733 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:45.910151 5733 raft_consensus.cc:1212] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.911410 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.913223 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.915791 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.916574 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.921262 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.923081 7798 tablet_copy_client.cc:538] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:45.923094 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.923906 7798 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:45.925344 5733 raft_consensus.cc:1212] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.926414 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c95203cf74b4b90867a9a3a334dbb21 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:45.926677 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.933843 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:45.934798 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.939327 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.940896 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:45.944629 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:45.944808 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:45.950806 5733 raft_consensus.cc:1212] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.951658 8550 tablet_replica.cc:331] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:45.951864 8550 raft_consensus.cc:2238] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:45.952009 8550 raft_consensus.cc:2267] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:45.952344 8550 ts_tablet_manager.cc:1905] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:45.955940 5733 raft_consensus.cc:1212] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.956152 7798 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:45.956377 7798 tablet_bootstrap.cc:492] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:45.957047 8550 ts_tablet_manager.cc:1918] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:45.957203 8550 log.cc:1198] T 3c95203cf74b4b90867a9a3a334dbb21 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/3c95203cf74b4b90867a9a3a334dbb21
I20250112 21:06:45.957635 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 3c95203cf74b4b90867a9a3a334dbb21 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
I20250112 21:06:45.957824 7798 ts_tablet_manager.cc:1397] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.034s user 0.005s sys 0.000s
I20250112 21:06:45.957962 7798 raft_consensus.cc:357] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.958047 7798 raft_consensus.cc:738] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:45.958109 7798 consensus_queue.cc:260] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:45.958339 7798 ts_tablet_manager.cc:1428] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:45.958519 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-c16b940a9e564da89ec84b46a6cf42d7 received from {username='slave'} at 127.26.82.71:49463
I20250112 21:06:45.958570 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-c16b940a9e564da89ec84b46a6cf42d7 on tablet c16b940a9e564da89ec84b46a6cf42d7 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.960510 5602 raft_consensus.cc:1212] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:45.972865 5470 raft_consensus.cc:1212] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.985605 5470 raft_consensus.cc:1212] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.990653 5470 raft_consensus.cc:1212] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:45.997179 5470 raft_consensus.cc:1212] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.005474 5733 raft_consensus.cc:1212] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.012574 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.013355 5864 raft_consensus.cc:1212] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.019656 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.022634 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.023928 5470 raft_consensus.cc:1212] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.030017 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.030826 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.031873 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.033219 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.038018 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.038281 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.038520 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.044801 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.044945 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.046037 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.049844 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.053061 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.053467 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.055759 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.058454 5864 raft_consensus.cc:1212] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.064410 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.064805 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.066803 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.069808 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.074807 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.077296 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.077502 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.087379 5602 raft_consensus.cc:1212] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.091497 5864 raft_consensus.cc:1212] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20250112 21:06:46.102557 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.107636 5733 raft_consensus.cc:1212] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.114078 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.114128 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.114805 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.114873 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.117974 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.120056 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.122802 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.124224 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.128389 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.128819 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.134260 5470 raft_consensus.cc:1212] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.135231 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.141193 5470 raft_consensus.cc:1212] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.144363 5864 raft_consensus.cc:1212] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.151294 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.152068 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.152115 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.164939 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.176527 5602 raft_consensus.cc:1212] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.187602 5864 raft_consensus.cc:1212] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.193312 5733 raft_consensus.cc:1212] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.206454 5602 raft_consensus.cc:1212] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.213238 4200 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:46.214049 5470 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.214416 3806 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.214654 6680 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.214876 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.215368 6664 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.215458 6664 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.217278 6664 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.218832 3661 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.218832 5864 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.219617 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:46.219830 8837 tablet_replica.cc:331] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:46.219931 8837 raft_consensus.cc:2238] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.220028 8837 raft_consensus.cc:2267] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.220294 8837 ts_tablet_manager.cc:1905] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.221526 5470 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.222148 8837 ts_tablet_manager.cc:1918] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:46.222204 8837 log.cc:1198] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:46.222570 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet de6b6ac6eeff431b9ac8cce47ea3a59e (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:46.228112 3806 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.242637 5864 raft_consensus.cc:1212] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.259615 5864 raft_consensus.cc:1212] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.259814 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.260140 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.263095 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.263356 5733 raft_consensus.cc:1212] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.270538 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.270803 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.277205 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.277590 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.279436 5470 raft_consensus.cc:1212] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.286206 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.292374 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.292496 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.293499 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.301878 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.302075 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.304127 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.304692 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.306128 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.308626 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.312755 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.321866 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.326128 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.328305 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.338964 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.339079 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.340086 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.340178 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.340279 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.344152 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.348341 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.348445 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.354279 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.355134 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.359045 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.379565 5470 raft_consensus.cc:1212] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.384335 5864 raft_consensus.cc:1212] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20250112 21:06:46.393517 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.394501 3806 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:46.395090 5602 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.395134 4200 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.395145 5733 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.395301 8743 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.395396 6648 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.396457 6648 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.397228 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.401296 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.403146 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.403169 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.407294 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.412339 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.414105 9054 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.414276 5733 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.415597 5602 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.415812 3663 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.416747 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b6276ae83948b6b4ea2771d9ce8355 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:46.416976 8837 tablet_replica.cc:331] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:46.417052 8837 raft_consensus.cc:2238] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.417110 8837 raft_consensus.cc:2267] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.417104 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.417337 8837 ts_tablet_manager.cc:1905] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.418687 8837 ts_tablet_manager.cc:1918] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:46.418732 8837 log.cc:1198] T b3b6276ae83948b6b4ea2771d9ce8355 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/b3b6276ae83948b6b4ea2771d9ce8355
I20250112 21:06:46.419828 4200 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.420447 3936 consensus_queue.cc:237] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:46.422084 4068 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.422192 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet b3b6276ae83948b6b4ea2771d9ce8355 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:46.422806 5733 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.422959 5470 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.423121 8535 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.423228 8535 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.424573 9065 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.425230 9065 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.426681 5733 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.427685 5470 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.428684 3663 catalog_manager.cc:5526] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.430310 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet c93763a09f7d4460b1874bf328fb06f5 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:46.430608 8550 tablet_replica.cc:331] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:46.430678 8550 raft_consensus.cc:2238] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.430742 8550 raft_consensus.cc:2267] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.430945 8550 ts_tablet_manager.cc:1905] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.432266 4068 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.434049 8550 ts_tablet_manager.cc:1918] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:46.434106 8550 log.cc:1198] T c93763a09f7d4460b1874bf328fb06f5 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/c93763a09f7d4460b1874bf328fb06f5
I20250112 21:06:46.434454 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet c93763a09f7d4460b1874bf328fb06f5 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:46.452535 3806 consensus_queue.cc:237] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:46.453184 5602 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.453217 4068 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.453430 9063 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.453567 9063 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.456813 9054 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.458318 4068 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.459331 3660 catalog_manager.cc:5526] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.460285 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ca3308b60c4367ab34f25ed5ecb0c0 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:46.462039 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.462222 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.464119 5602 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.470219 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.472533 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.474550 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.479877 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.482941 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.483230 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.463974 9175 tablet_replica.cc:331] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:46.484027 9175 raft_consensus.cc:2238] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.484163 9175 raft_consensus.cc:2267] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.484541 9175 ts_tablet_manager.cc:1905] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.486455 9175 ts_tablet_manager.cc:1918] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:46.486985 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.492465 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.492796 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.492388 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.493315 9175 log.cc:1198] T 47ca3308b60c4367ab34f25ed5ecb0c0 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/47ca3308b60c4367ab34f25ed5ecb0c0
I20250112 21:06:46.494392 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.494592 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 47ca3308b60c4367ab34f25ed5ecb0c0 (table rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]) successfully deleted
I20250112 21:06:46.500383 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.500460 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.512020 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.534600 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.539701 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.540480 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.544742 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.546977 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.549563 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.551473 3806 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a"
dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36844
I20250112 21:06:46.551573 3806 raft_consensus.cc:604] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:46.554860 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.555135 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.559092 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.567459 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.574147 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.574363 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.575271 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.584498 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.586402 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.592442 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.610260 3806 consensus_queue.cc:237] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:46.610827 5864 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.610983 5470 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.611352 9054 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.611471 9054 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.611518 4068 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.612336 6648 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.615296 6648 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:46.616494 5864 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:46.618255 4068 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:46.619504 3665 catalog_manager.cc:5526] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:46.620136 5470 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:46.621773 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e0dcaf59aa4303aff6810c70092184 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:46.621908 9175 tablet_replica.cc:331] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:46.621990 9175 raft_consensus.cc:2238] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.622062 9175 raft_consensus.cc:2267] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.622303 9175 ts_tablet_manager.cc:1905] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.623589 9175 ts_tablet_manager.cc:1918] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:46.623641 9175 log.cc:1198] T 56e0dcaf59aa4303aff6810c70092184 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/56e0dcaf59aa4303aff6810c70092184
I20250112 21:06:46.623998 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 56e0dcaf59aa4303aff6810c70092184 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:46.625703 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.630291 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.638020 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.641335 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.641624 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.643414 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.646951 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.651428 5713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.653810 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.654088 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.654390 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.658499 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.658885 3787 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:46.659653 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.662015 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.671111 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.672910 4200 consensus_queue.cc:237] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:46.673429 5602 raft_consensus.cc:1270] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.673444 3806 raft_consensus.cc:1270] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.673629 8822 consensus_queue.cc:1035] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.673779 8822 consensus_queue.cc:1035] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.674571 6680 raft_consensus.cc:2949] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.674886 3806 raft_consensus.cc:2949] T 1fb3deb52b8c4cceb1814858ea30e94f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.674903 5602 raft_consensus.cc:2949] T 1fb3deb52b8c4cceb1814858ea30e94f P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.676071 3663 catalog_manager.cc:5526] T 1fb3deb52b8c4cceb1814858ea30e94f P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.677356 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fb3deb52b8c4cceb1814858ea30e94f with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:46.677459 9175 tablet_replica.cc:331] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:46.677523 9175 raft_consensus.cc:2238] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.677570 9175 raft_consensus.cc:2267] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.677749 9175 ts_tablet_manager.cc:1905] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.679088 9175 ts_tablet_manager.cc:1918] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:46.679147 9175 log.cc:1198] T 1fb3deb52b8c4cceb1814858ea30e94f P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/1fb3deb52b8c4cceb1814858ea30e94f
I20250112 21:06:46.681243 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 1fb3deb52b8c4cceb1814858ea30e94f (table rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]) successfully deleted
I20250112 21:06:46.686460 9063 raft_consensus.cc:988] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: : Instructing follower 85a6e2a0c3b94dea8a094d396cfcac4b to start an election
I20250112 21:06:46.686522 9063 raft_consensus.cc:1076] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Signalling peer 85a6e2a0c3b94dea8a094d396cfcac4b to start an election
I20250112 21:06:46.686720 5733 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a"
dest_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
from {username='slave'} at 127.26.82.65:37143
I20250112 21:06:46.686857 5733 raft_consensus.cc:491] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:46.686915 5733 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:46.687476 5733 raft_consensus.cc:513] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:46.687786 5733 leader_election.cc:290] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [CANDIDATE]: Term 2 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:46.688223 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:46.688297 4200 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:46.689045 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:46.689129 3806 raft_consensus.cc:3049] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:46.689174 3806 raft_consensus.cc:738] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:46.689226 3806 consensus_queue.cc:260] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:46.689327 3806 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:46.689934 3806 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85a6e2a0c3b94dea8a094d396cfcac4b in term 2.
I20250112 21:06:46.690280 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:46.690354 3936 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:46.691043 3936 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85a6e2a0c3b94dea8a094d396cfcac4b in term 2.
I20250112 21:06:46.691475 5667 leader_election.cc:304] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 85a6e2a0c3b94dea8a094d396cfcac4b, bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:46.691941 7563 raft_consensus.cc:2798] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:46.692051 7563 raft_consensus.cc:695] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Becoming Leader. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Running, Role: LEADER
I20250112 21:06:46.692132 7563 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:46.692194 4200 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85a6e2a0c3b94dea8a094d396cfcac4b in term 2.
I20250112 21:06:46.693554 3658 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: term changed from 1 to 2, leader changed from bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) to 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71). New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.699358 5470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345f4bf45767475ca481525f3d26a34a" candidate_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "6f7f47d877a8407185013dba2959fddd"
I20250112 21:06:46.699457 5470 raft_consensus.cc:3054] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:46.700342 5470 raft_consensus.cc:2463] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85a6e2a0c3b94dea8a094d396cfcac4b in term 2.
I20250112 21:06:46.708535 3936 consensus_queue.cc:237] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:46.709093 4200 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.709334 5470 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.709388 9065 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.709874 9065 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.710232 9065 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.711477 4200 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.712052 3656 catalog_manager.cc:5526] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.714679 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.715317 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fae2c2254d04e7a819f5d8638fd97ea with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:46.715441 8550 tablet_replica.cc:331] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:46.715497 8550 raft_consensus.cc:2238] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.715539 8550 raft_consensus.cc:2267] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.715745 8550 ts_tablet_manager.cc:1905] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.717108 8550 ts_tablet_manager.cc:1918] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:46.717157 8550 log.cc:1198] T 3fae2c2254d04e7a819f5d8638fd97ea P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/3fae2c2254d04e7a819f5d8638fd97ea
I20250112 21:06:46.717926 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 3fae2c2254d04e7a819f5d8638fd97ea (table rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]) successfully deleted
I20250112 21:06:46.718017 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.725939 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.733976 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:46.736683 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:46.737352 5470 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.738232 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.740725 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.747141 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.749850 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.755802 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.759276 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:46.765723 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.770277 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:46.770859 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.774013 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.774387 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.778501 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.779516 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:46.784516 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.793241 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:46.797798 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.798393 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.799898 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.808933 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.823057 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.828594 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.834785 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.835395 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:46.839455 4200 consensus_queue.cc:237] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:46.839910 5470 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.840230 8822 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.840319 4068 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:46.840538 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.840927 8822 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:46.842149 8822 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.842412 5470 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.843820 3665 catalog_manager.cc:5526] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.844429 4068 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.845422 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 979cf479af044430904e610adee50359 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:46.845651 8550 tablet_replica.cc:331] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:46.845718 8550 raft_consensus.cc:2238] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.845762 8550 raft_consensus.cc:2267] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.845914 8550 ts_tablet_manager.cc:1905] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.847673 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:46.848060 8550 ts_tablet_manager.cc:1918] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:46.848109 8550 log.cc:1198] T 979cf479af044430904e610adee50359 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/979cf479af044430904e610adee50359
I20250112 21:06:46.848433 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 979cf479af044430904e610adee50359 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
I20250112 21:06:46.854003 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.859040 4200 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:46.859664 5733 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.859712 4068 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.859726 5602 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.860152 9643 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.860420 8822 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.860518 8822 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.861640 9643 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.861801 5602 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.862915 5733 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.863659 3658 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.864086 4068 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:46.864562 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 87835ddc19954987b0f17f6aa0370c10 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:46.864702 9175 tablet_replica.cc:331] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:46.864770 9175 raft_consensus.cc:2238] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.864850 9175 raft_consensus.cc:2267] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.865022 9175 ts_tablet_manager.cc:1905] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.866384 9175 ts_tablet_manager.cc:1918] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:46.866441 9175 log.cc:1198] T 87835ddc19954987b0f17f6aa0370c10 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:46.867664 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 87835ddc19954987b0f17f6aa0370c10 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:46.869592 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.871013 4068 consensus_queue.cc:237] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:46.872117 4200 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.872120 5864 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.872128 5470 raft_consensus.cc:1270] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.872738 9682 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.872859 9682 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.872936 9682 consensus_queue.cc:1035] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.875886 6877 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.877064 4200 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.877310 5864 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.877966 3656 catalog_manager.cc:5526] T 0bad642a13544a3caa0ca9bcaa4a1257 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.877990 5470 raft_consensus.cc:2949] T 0bad642a13544a3caa0ca9bcaa4a1257 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:46.879479 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bad642a13544a3caa0ca9bcaa4a1257 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:46.879585 9175 tablet_replica.cc:331] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:46.879658 9175 raft_consensus.cc:2238] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:46.879731 9175 raft_consensus.cc:2267] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:46.879885 9175 ts_tablet_manager.cc:1905] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:46.881250 9175 ts_tablet_manager.cc:1918] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:46.882110 9175 log.cc:1198] T 0bad642a13544a3caa0ca9bcaa4a1257 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/0bad642a13544a3caa0ca9bcaa4a1257
I20250112 21:06:46.882483 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 0bad642a13544a3caa0ca9bcaa4a1257 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:46.899320 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.908171 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.908334 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.914119 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.916735 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.921411 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:46.928762 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.929488 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.931890 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:46.936049 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.941074 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.945618 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.946719 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:46.955320 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.959285 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:46.959815 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.960533 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.963038 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.979185 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:46.982784 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.982805 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:46.988839 3938 consensus_queue.cc:237] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:46.989414 4068 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.989431 5733 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:46.989601 9066 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.989673 9065 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:46.991441 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:46.992094 9066 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.993610 5733 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.993696 3660 catalog_manager.cc:5526] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.994704 4068 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:46.995694 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 90f8eabb6a0d4292953e3dec2efec97d with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:46.997530 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.998660 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:46.999348 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.000119 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.000519 9860 tablet_replica.cc:331] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:47.002895 5470 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.006453 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.010419 9065 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.011101 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.011068 5470 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.011973 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.012228 9860 raft_consensus.cc:2238] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.012334 9860 raft_consensus.cc:2267] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.012588 9860 ts_tablet_manager.cc:1905] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.012866 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.013379 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.018543 9860 ts_tablet_manager.cc:1918] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:47.018661 9860 log.cc:1198] T 90f8eabb6a0d4292953e3dec2efec97d P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/90f8eabb6a0d4292953e3dec2efec97d
I20250112 21:06:47.019106 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 90f8eabb6a0d4292953e3dec2efec97d (table rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]) successfully deleted
I20250112 21:06:47.020022 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.030407 3806 consensus_queue.cc:237] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:47.031030 4068 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.031232 8743 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.031646 5470 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.031721 5733 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.031911 8743 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.032558 9063 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.032850 8743 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.032985 5470 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.034085 4068 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.035319 3656 catalog_manager.cc:5526] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.036252 5733 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.037370 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 917b9e7ed0ed43efa0a60ff461e4f213 with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:47.037621 9860 tablet_replica.cc:331] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:47.037691 9860 raft_consensus.cc:2238] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.037757 9860 raft_consensus.cc:2267] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.037959 9860 ts_tablet_manager.cc:1905] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.039268 9860 ts_tablet_manager.cc:1918] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:47.039314 9860 log.cc:1198] T 917b9e7ed0ed43efa0a60ff461e4f213 P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/917b9e7ed0ed43efa0a60ff461e4f213
I20250112 21:06:47.039645 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 917b9e7ed0ed43efa0a60ff461e4f213 (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:47.040807 3938 consensus_queue.cc:237] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:47.041299 3806 raft_consensus.cc:1270] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.041371 5733 raft_consensus.cc:1270] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.041553 9859 consensus_queue.cc:1035] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.041668 9065 consensus_queue.cc:1035] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.042454 9859 raft_consensus.cc:2949] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.043318 5733 raft_consensus.cc:2949] T e82a54be68bc4c3ca23a516694e980eb P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.043566 3806 raft_consensus.cc:2949] T e82a54be68bc4c3ca23a516694e980eb P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.045064 3660 catalog_manager.cc:5526] T e82a54be68bc4c3ca23a516694e980eb P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.047427 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet e82a54be68bc4c3ca23a516694e980eb with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:47.047551 9860 tablet_replica.cc:331] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:47.047603 9860 raft_consensus.cc:2238] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.047647 9860 raft_consensus.cc:2267] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.047801 9860 ts_tablet_manager.cc:1905] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.049178 9860 ts_tablet_manager.cc:1918] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.049228 9860 log.cc:1198] T e82a54be68bc4c3ca23a516694e980eb P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/e82a54be68bc4c3ca23a516694e980eb
I20250112 21:06:47.049542 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet e82a54be68bc4c3ca23a516694e980eb (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:47.062767 3938 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:47.063037 3806 consensus_queue.cc:237] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.063437 5602 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.063634 4068 raft_consensus.cc:1270] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.063844 9063 consensus_queue.cc:1035] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.063911 5864 raft_consensus.cc:1270] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.064253 9063 consensus_queue.cc:1035] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.064254 9065 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.065145 9063 raft_consensus.cc:2949] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.065253 4068 raft_consensus.cc:2949] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.065960 4067 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.066447 3656 catalog_manager.cc:5526] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.066617 9065 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.066840 5864 raft_consensus.cc:2949] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.067816 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:47.067912 9860 tablet_replica.cc:331] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:47.067961 9860 raft_consensus.cc:2238] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.068002 9860 raft_consensus.cc:2267] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.068143 9860 ts_tablet_manager.cc:1905] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.068156 9859 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.068214 5602 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.068636 4067 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.069370 9860 ts_tablet_manager.cc:1918] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.069416 9860 log.cc:1198] T bc53f2e07c4746dcb4b0ddcb8791a4d7 P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/bc53f2e07c4746dcb4b0ddcb8791a4d7
I20250112 21:06:47.069770 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet bc53f2e07c4746dcb4b0ddcb8791a4d7 (table rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]) successfully deleted
I20250112 21:06:47.070672 3658 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.072147 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:47.072275 9860 tablet_replica.cc:331] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:47.072328 9860 raft_consensus.cc:2238] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.072391 9860 raft_consensus.cc:2267] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.072548 9860 ts_tablet_manager.cc:1905] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.073755 9860 ts_tablet_manager.cc:1918] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.073799 9860 log.cc:1198] T 2e6bf2e45872401f8245a586b6fe7431 P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:47.074110 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 2e6bf2e45872401f8245a586b6fe7431 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:47.103080 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.103827 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.114841 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.115470 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.115526 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.119213 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.119750 3938 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:47.119976 9434 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.124176 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.124518 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.127115 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.127331 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.134368 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.136804 5470 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:47.136998 9434 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.139416 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.146265 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.153018 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.155212 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.155606 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.155864 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.161285 3806 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:47.161433 4200 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:47.161507 7563 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.161773 7563 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.162235 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.164491 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.172210 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.175458 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.178411 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.184464 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.185768 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.186537 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.191213 3917 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.191264 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.191668 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.197273 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.202581 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.204490 3806 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.205014 5864 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.205034 5602 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.205210 9054 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.205279 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.205379 9054 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.205859 9054 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.206465 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.208339 9054 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.209499 5864 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.210577 3660 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.210686 5602 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.211357 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 8880a7467c6a44da96f98c3cc368c9f0 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:47.212785 10159 tablet_replica.cc:331] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:47.212920 10159 raft_consensus.cc:2238] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.213160 10159 raft_consensus.cc:2267] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.213390 10159 ts_tablet_manager.cc:1905] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.215214 3936 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.216800 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.228721 10159 ts_tablet_manager.cc:1918] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:47.228830 10159 log.cc:1198] T 8880a7467c6a44da96f98c3cc368c9f0 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:47.229430 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 8880a7467c6a44da96f98c3cc368c9f0 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:47.245023 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.245463 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.249253 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.251299 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.254256 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.255451 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.256413 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.260669 3936 consensus_queue.cc:237] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:47.260936 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.261580 5733 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:47.261852 5470 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:47.261971 5733 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:47.262441 9859 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:47.262526 5470 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:47.262661 4199 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:47.262740 3806 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:47.262740 9434 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:47.262939 9859 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:47.263108 9434 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:47.263214 9434 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:47.264596 10117 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.264763 5470 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.265987 3664 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 5 to 7, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.266002 3806 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.266239 9859 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.266284 4200 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.267374 5470 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.267668 5733 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.268756 3916 tablet_service.cc:1514] Processing DeleteTablet for tablet 345f4bf45767475ca481525f3d26a34a with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:47.268769 3661 catalog_manager.cc:5526] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.268944 9175 tablet_replica.cc:331] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:47.269018 9175 raft_consensus.cc:2238] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.269089 9175 raft_consensus.cc:2267] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.270696 9175 ts_tablet_manager.cc:1905] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.271708 3806 consensus_queue.cc:237] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.272079 5470 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:47.272096 9175 ts_tablet_manager.cc:1918] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20250112 21:06:47.272100 5864 raft_consensus.cc:1270] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:47.272142 9175 log.cc:1198] T 345f4bf45767475ca481525f3d26a34a P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/345f4bf45767475ca481525f3d26a34a
I20250112 21:06:47.272240 8743 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:47.272323 9063 consensus_queue.cc:1035] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:47.272565 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 345f4bf45767475ca481525f3d26a34a (table rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]) successfully deleted
I20250112 21:06:47.273576 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet c93763a09f7d4460b1874bf328fb06f5 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:47.273736 10159 tablet_replica.cc:331] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:47.273798 10159 raft_consensus.cc:2238] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.273867 10159 raft_consensus.cc:2267] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.274036 8743 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.274224 10159 ts_tablet_manager.cc:1905] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.275028 5470 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.275211 3664 catalog_manager.cc:5526] T 56e0dcaf59aa4303aff6810c70092184 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.276111 10159 ts_tablet_manager.cc:1918] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:47.276163 10159 log.cc:1198] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/c93763a09f7d4460b1874bf328fb06f5
I20250112 21:06:47.276322 5864 raft_consensus.cc:2949] T 56e0dcaf59aa4303aff6810c70092184 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.277381 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet c93763a09f7d4460b1874bf328fb06f5 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:47.277468 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e0dcaf59aa4303aff6810c70092184 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:47.277642 10159 tablet_replica.cc:331] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:47.277699 10159 raft_consensus.cc:2238] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.277756 10159 raft_consensus.cc:2267] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.278066 10159 ts_tablet_manager.cc:1905] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.279075 10159 ts_tablet_manager.cc:1918] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:47.279130 10159 log.cc:1198] T 56e0dcaf59aa4303aff6810c70092184 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/56e0dcaf59aa4303aff6810c70092184
I20250112 21:06:47.279855 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 56e0dcaf59aa4303aff6810c70092184 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:47.305634 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.310357 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.313201 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.313302 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.318423 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.323487 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.325856 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.329875 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.352636 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.357172 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.360936 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.362628 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.363441 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.374912 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.374948 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.382203 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.386370 4067 consensus_queue.cc:237] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:47.387060 3936 raft_consensus.cc:1270] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.387497 5602 raft_consensus.cc:1270] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.387795 10434 consensus_queue.cc:1035] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.388476 10434 raft_consensus.cc:2949] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.389508 9682 consensus_queue.cc:1035] T 7493a3e50b0e4fce955005b60a713348 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.389688 3936 raft_consensus.cc:2949] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.390035 5602 raft_consensus.cc:2949] T 7493a3e50b0e4fce955005b60a713348 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.391386 3660 catalog_manager.cc:5526] T 7493a3e50b0e4fce955005b60a713348 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 4 to 5, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.392911 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 7493a3e50b0e4fce955005b60a713348 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:47.393433 10447 tablet_replica.cc:331] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:47.393498 10447 raft_consensus.cc:2238] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.393543 10447 raft_consensus.cc:2267] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.393775 10447 ts_tablet_manager.cc:1905] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.395407 10447 ts_tablet_manager.cc:1918] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.395512 10447 log.cc:1198] T 7493a3e50b0e4fce955005b60a713348 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/7493a3e50b0e4fce955005b60a713348
I20250112 21:06:47.395882 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 7493a3e50b0e4fce955005b60a713348 (table rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]) successfully deleted
I20250112 21:06:47.404388 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.414041 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.415475 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.415740 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.417815 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.423048 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.425066 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.430137 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.433843 4199 consensus_queue.cc:237] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.434365 5864 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.434585 4067 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.434624 6680 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.435241 5470 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.435367 8822 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.435663 6680 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.441054 8822 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.442425 5864 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.443332 5470 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.443655 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.444254 3661 catalog_manager.cc:5526] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.446012 4067 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.447796 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ba48c8dc7e4ee485f85ee94aca256b with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:47.449360 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.450505 10447 tablet_replica.cc:331] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:47.450577 10447 raft_consensus.cc:2238] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.450631 10447 raft_consensus.cc:2267] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.450856 10447 ts_tablet_manager.cc:1905] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.451567 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.452152 10447 ts_tablet_manager.cc:1918] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:47.452196 10447 log.cc:1198] T b6ba48c8dc7e4ee485f85ee94aca256b P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/b6ba48c8dc7e4ee485f85ee94aca256b
I20250112 21:06:47.453244 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.455250 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet b6ba48c8dc7e4ee485f85ee94aca256b (table rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]) successfully deleted
I20250112 21:06:47.462649 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.463302 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.470956 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.471055 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.471272 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.472532 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.478019 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.480058 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.480809 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.495915 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.497385 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.503947 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.518926 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.523205 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.529520 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.535893 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.539630 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.539860 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.542313 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.544471 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.547861 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.551384 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.554833 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.563642 4067 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.563730 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.563977 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.564033 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.564108 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.564126 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.564487 5864 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.564635 5733 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.564777 4199 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.564877 9682 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.565009 9682 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.565094 9682 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.573906 10434 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.575217 5733 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.575302 3661 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.576826 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 7249078cad3e4bc3ab2b62d8887987df with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:47.576927 10447 tablet_replica.cc:331] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:47.576861 5864 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.577000 10447 raft_consensus.cc:2238] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.577081 10447 raft_consensus.cc:2267] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.577252 10447 ts_tablet_manager.cc:1905] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.577533 4199 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.578840 10447 ts_tablet_manager.cc:1918] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:47.578898 10447 log.cc:1198] T 7249078cad3e4bc3ab2b62d8887987df P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:47.579221 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 7249078cad3e4bc3ab2b62d8887987df (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:47.597589 3936 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:47.598147 5602 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.597618 3938 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:47.598178 4067 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.598408 9065 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.598558 3806 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.598688 8535 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20250112 21:06:47.598793 3873 consensus_peers.cc:487] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e6bf2e45872401f8245a586b6fe7431. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:47.598798 8535 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.598937 5733 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.599483 8535 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.600129 8535 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.600214 5733 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.601047 8535 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.602067 3663 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.602241 4067 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.603355 5602 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.604187 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:47.604280 10159 tablet_replica.cc:331] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:47.604338 10159 raft_consensus.cc:2238] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.604384 10159 raft_consensus.cc:2267] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.604591 10159 ts_tablet_manager.cc:1905] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.606062 10159 ts_tablet_manager.cc:1918] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.606112 10159 log.cc:1198] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:47.606482 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 0d2ae40fed4b41218aa58c523d7d9785 (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:47.605170 3661 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 5 to 6, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:47.606719 3938 consensus_queue.cc:237] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:47.607371 4199 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.607555 5602 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.607730 9859 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.607715 3806 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.607797 9065 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.608179 5733 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.608397 9859 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.609117 9065 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.609344 5733 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.610853 5602 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.610922 3938 consensus_queue.cc:237] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.611891 3656 catalog_manager.cc:5526] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.611985 4199 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.612167 3806 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.612175 5864 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.612190 5470 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.612349 9859 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.612450 9859 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.612645 9859 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.613191 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet c16b940a9e564da89ec84b46a6cf42d7 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:47.613282 10447 tablet_replica.cc:331] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:47.613341 10447 raft_consensus.cc:2238] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.613394 10447 raft_consensus.cc:2267] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.613546 10447 ts_tablet_manager.cc:1905] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.614087 9859 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.614738 10447 ts_tablet_manager.cc:1918] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:47.614773 10447 log.cc:1198] T c16b940a9e564da89ec84b46a6cf42d7 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/c16b940a9e564da89ec84b46a6cf42d7
I20250112 21:06:47.614944 5864 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.616474 5470 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.617015 3656 catalog_manager.cc:5526] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.617450 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet c16b940a9e564da89ec84b46a6cf42d7 (table rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]) successfully deleted
I20250112 21:06:47.618316 3806 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.620473 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fce4710d0784d02991a6465522323d1 with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:47.620695 10755 tablet_replica.cc:331] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:47.620829 10755 raft_consensus.cc:2238] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.620946 10755 raft_consensus.cc:2267] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.621162 10755 ts_tablet_manager.cc:1905] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.622668 10755 ts_tablet_manager.cc:1918] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:47.622792 10755 log.cc:1198] T 0fce4710d0784d02991a6465522323d1 P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/0fce4710d0784d02991a6465522323d1
I20250112 21:06:47.623446 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 0fce4710d0784d02991a6465522323d1 (table rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]) successfully deleted
I20250112 21:06:47.633838 3938 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.634649 5733 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20250112 21:06:47.634770 3872 consensus_peers.cc:487] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c95203cf74b4b90867a9a3a334dbb21. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:47.634786 4067 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:47.635048 9859 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.635727 9859 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:47.641331 9859 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.642635 4067 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.642716 3661 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 5 to 6, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.647063 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.647248 5733 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.649678 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.652312 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.658149 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.668166 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.669020 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.672712 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.672978 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.700990 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.701278 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.701313 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.707911 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.712975 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.713277 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.717262 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.718737 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.724210 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.728814 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.728984 4199 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f"
dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46214
I20250112 21:06:47.729091 4199 raft_consensus.cc:604] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:47.730508 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.736085 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.742336 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.745666 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.746460 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.746593 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.746740 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.746963 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.750800 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.754207 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.761250 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.765103 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.765707 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.770203 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.794291 4067 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.794910 3938 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.795403 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.795428 9682 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.795655 9682 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.796253 9682 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.797798 3938 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.799437 3660 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.800307 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 102b9dd051ff4a2a8da8ff4692e56256 with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:47.800437 10755 tablet_replica.cc:331] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:47.800518 10755 raft_consensus.cc:2238] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.800454 5864 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.800576 10755 raft_consensus.cc:2267] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.800747 4199 consensus_queue.cc:237] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:47.800854 10755 ts_tablet_manager.cc:1905] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.801196 4067 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.801223 5602 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.801537 10518 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.801678 10518 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.802287 10755 ts_tablet_manager.cc:1918] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.802340 10755 log.cc:1198] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:47.802845 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 102b9dd051ff4a2a8da8ff4692e56256 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:47.803227 10511 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.804320 5602 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.804428 4067 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.805514 3657 catalog_manager.cc:5526] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:47.806285 3916 tablet_service.cc:1514] Processing DeleteTablet for tablet dd01159cf53642739a8a1dbcacbc933d with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:47.809890 11043 tablet_replica.cc:331] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:47.810001 11043 raft_consensus.cc:2238] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.810068 11043 raft_consensus.cc:2267] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.810022 3806 consensus_queue.cc:237] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.810243 11043 ts_tablet_manager.cc:1905] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.810775 4199 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.811214 9054 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.811774 5864 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.811911 11043 ts_tablet_manager.cc:1918] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.811964 11043 log.cc:1198] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/dd01159cf53642739a8a1dbcacbc933d
I20250112 21:06:47.811960 9054 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.812335 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet dd01159cf53642739a8a1dbcacbc933d (table rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]) successfully deleted
I20250112 21:06:47.813091 9054 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.813211 4199 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.814312 3660 catalog_manager.cc:5526] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.814980 5864 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.815148 3916 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9176ff75364d81a5b3bdd0e7e10673 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:47.815240 11043 tablet_replica.cc:331] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:47.815308 11043 raft_consensus.cc:2238] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.815382 11043 raft_consensus.cc:2267] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.815568 11043 ts_tablet_manager.cc:1905] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.816902 11043 ts_tablet_manager.cc:1918] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.816957 11043 log.cc:1198] T dd9176ff75364d81a5b3bdd0e7e10673 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/dd9176ff75364d81a5b3bdd0e7e10673
I20250112 21:06:47.817533 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet dd9176ff75364d81a5b3bdd0e7e10673 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
I20250112 21:06:47.822331 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.823635 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.831184 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.832320 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.832408 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.839601 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.843811 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.844522 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.852789 4178 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.856005 4049 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:47.858079 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.861176 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.864614 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.865005 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.881356 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.883116 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.890698 4199 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e"
dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46308
I20250112 21:06:47.890777 4199 raft_consensus.cc:604] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:47.891505 10518 raft_consensus.cc:988] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: : Instructing follower 2565f97e9cc54700ab2ce39aa7a546f8 to start an election
I20250112 21:06:47.891564 10518 raft_consensus.cc:1076] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Signalling peer 2565f97e9cc54700ab2ce39aa7a546f8 to start an election
I20250112 21:06:47.891755 4067 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f"
dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
from {username='slave'} at 127.26.82.68:58711
I20250112 21:06:47.891824 4067 raft_consensus.cc:491] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:47.891850 4067 raft_consensus.cc:3054] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:47.892809 4067 raft_consensus.cc:513] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.892969 4067 leader_election.cc:290] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:47.893213 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:47.893282 4199 raft_consensus.cc:3049] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:47.893304 4199 raft_consensus.cc:738] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:47.893266 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
I20250112 21:06:47.893299 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5da4fd48e1f24680babd4f1874b31b5f" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:47.893342 4199 consensus_queue.cc:260] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.893365 3806 raft_consensus.cc:3054] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:47.893353 5864 raft_consensus.cc:3054] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:47.893410 4199 raft_consensus.cc:3054] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:47.894058 3806 raft_consensus.cc:2463] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:47.894057 5864 raft_consensus.cc:2463] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:47.894222 4199 raft_consensus.cc:2463] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:47.894295 4003 leader_election.cc:304] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, 38c8e8afd3f5411490f56ebf57aee109, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:47.894508 10642 raft_consensus.cc:2798] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:47.894582 10642 raft_consensus.cc:695] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Becoming Leader. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:47.894629 10642 consensus_queue.cc:237] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:47.895934 3658 catalog_manager.cc:5526] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: term changed from 1 to 2, leader changed from 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) to 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67). New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:47.902879 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:47.905202 4178 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:47.906308 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.908733 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.915630 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.918058 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.919464 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.924330 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.958117 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.967046 4178 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:47.970600 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:47.974187 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.981549 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:47.981979 4199 consensus_queue.cc:237] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:47.982411 5733 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.982590 6680 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.983481 6680 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.984776 5733 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.984865 3938 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20250112 21:06:47.985181 10518 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:47.985827 3660 catalog_manager.cc:5526] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.986719 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ff25b3286e45af8ddd38965062fca5 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:47.986835 10159 tablet_replica.cc:331] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:47.986903 10159 raft_consensus.cc:2238] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:47.987049 10159 raft_consensus.cc:2267] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:47.987247 10159 ts_tablet_manager.cc:1905] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:47.987612 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:47.987882 3938 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:47.988688 10159 ts_tablet_manager.cc:1918] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20250112 21:06:47.988732 10159 log.cc:1198] T 74ff25b3286e45af8ddd38965062fca5 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/74ff25b3286e45af8ddd38965062fca5
I20250112 21:06:47.989429 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:47.989477 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 74ff25b3286e45af8ddd38965062fca5 (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:47.990384 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:47.992205 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:47.997687 4178 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.001250 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.005259 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.006531 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.008275 4178 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.013206 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.013989 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:48.014110 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.014492 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.017207 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.020063 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.022231 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:48.024874 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.026226 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.043853 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.045109 3936 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0fce4710d0784d02991a6465522323d1"
dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54630
I20250112 21:06:48.045190 3936 raft_consensus.cc:604] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:48.054893 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.059881 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.062992 4178 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.065506 5451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.066617 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.068246 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:48.072696 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.074031 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.088181 4067 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.088703 5733 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:48.088762 5864 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:48.088961 9682 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.089131 9682 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.089567 10642 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.090294 5733 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.091030 5864 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.091531 3664 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.093017 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 7249078cad3e4bc3ab2b62d8887987df with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:48.093204 10755 tablet_replica.cc:331] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:48.093268 10755 raft_consensus.cc:2238] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:48.093333 10755 raft_consensus.cc:2267] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:48.093719 10755 ts_tablet_manager.cc:1905] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:48.095520 10755 ts_tablet_manager.cc:1918] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:48.095577 10755 log.cc:1198] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:48.095966 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 7249078cad3e4bc3ab2b62d8887987df (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:48.107656 10518 raft_consensus.cc:988] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: : Instructing follower 6f7f47d877a8407185013dba2959fddd to start an election
I20250112 21:06:48.107728 10518 raft_consensus.cc:1076] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Signalling peer 6f7f47d877a8407185013dba2959fddd to start an election
I20250112 21:06:48.108027 5470 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e"
dest_uuid: "6f7f47d877a8407185013dba2959fddd"
from {username='slave'} at 127.26.82.68:59637
I20250112 21:06:48.108134 5470 raft_consensus.cc:491] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:48.108162 5470 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.108995 5470 raft_consensus.cc:513] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:48.109295 5470 leader_election.cc:290] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [CANDIDATE]: Term 2 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:48.109404 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "6f7f47d877a8407185013dba2959fddd" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:48.109470 4199 raft_consensus.cc:3049] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:48.109493 4199 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:48.109541 4199 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:48.109622 4199 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.109632 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "6f7f47d877a8407185013dba2959fddd" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:48.109716 3806 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.110306 4199 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7f47d877a8407185013dba2959fddd in term 2.
I20250112 21:06:48.110394 3806 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7f47d877a8407185013dba2959fddd in term 2.
I20250112 21:06:48.110581 5405 leader_election.cc:304] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 6f7f47d877a8407185013dba2959fddd, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:48.110751 7597 raft_consensus.cc:2798] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:48.110877 7597 raft_consensus.cc:695] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Becoming Leader. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Running, Role: LEADER
I20250112 21:06:48.110955 7597 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:48.111773 3663 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd reported cstate change: term changed from 1 to 2, leader changed from 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) to 6f7f47d877a8407185013dba2959fddd (127.26.82.69). New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:48.117686 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "6f7f47d877a8407185013dba2959fddd" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
I20250112 21:06:48.117769 5864 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.118496 5864 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7f47d877a8407185013dba2959fddd in term 2.
I20250112 21:06:48.126029 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.134171 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.134621 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.141386 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.145416 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.146152 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.151384 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:48.152117 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.159010 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.159389 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:48.160167 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.162184 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.162322 11536 ts_tablet_manager.cc:927] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:48.162581 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.162667 11536 tablet_copy_client.cc:323] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:48.162843 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 2e6bf2e45872401f8245a586b6fe7431 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:53179)
I20250112 21:06:48.162889 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 2e6bf2e45872401f8245a586b6fe7431 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:53179: session id = 6f7f47d877a8407185013dba2959fddd-2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:48.163410 3958 tablet_copy_source_session.cc:215] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:48.166503 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.171396 11536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6bf2e45872401f8245a586b6fe7431. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:48.172938 11536 tablet_copy_client.cc:806] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:48.173067 11536 tablet_copy_client.cc:670] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:48.175869 11536 tablet_copy_client.cc:538] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:48.176534 11536 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:48.178196 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.179291 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.180920 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.183239 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.184345 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.188040 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.189998 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.191521 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.196239 3916 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20250112 21:06:48.196564 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.209335 10717 raft_consensus.cc:988] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: : Instructing follower 6f7f47d877a8407185013dba2959fddd to start an election
I20250112 21:06:48.209398 10717 raft_consensus.cc:1076] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Signalling peer 6f7f47d877a8407185013dba2959fddd to start an election
I20250112 21:06:48.209646 5470 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0fce4710d0784d02991a6465522323d1"
dest_uuid: "6f7f47d877a8407185013dba2959fddd"
from {username='slave'} at 127.26.82.66:54803
I20250112 21:06:48.209729 5470 raft_consensus.cc:491] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:48.209764 5470 raft_consensus.cc:3054] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.210492 5470 raft_consensus.cc:513] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.210657 5470 leader_election.cc:290] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [CANDIDATE]: Term 2 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:48.210803 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fce4710d0784d02991a6465522323d1" candidate_uuid: "6f7f47d877a8407185013dba2959fddd" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:48.210871 3936 raft_consensus.cc:3049] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:48.210841 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fce4710d0784d02991a6465522323d1" candidate_uuid: "6f7f47d877a8407185013dba2959fddd" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:48.210898 3936 raft_consensus.cc:738] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:48.210918 3806 raft_consensus.cc:3054] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.210924 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fce4710d0784d02991a6465522323d1" candidate_uuid: "6f7f47d877a8407185013dba2959fddd" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
I20250112 21:06:48.210991 5864 raft_consensus.cc:3054] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.210954 3936 consensus_queue.cc:260] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.211048 3936 raft_consensus.cc:3054] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.211659 5864 raft_consensus.cc:2463] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7f47d877a8407185013dba2959fddd in term 2.
I20250112 21:06:48.211678 3806 raft_consensus.cc:2463] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7f47d877a8407185013dba2959fddd in term 2.
I20250112 21:06:48.211721 3936 raft_consensus.cc:2463] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7f47d877a8407185013dba2959fddd in term 2.
I20250112 21:06:48.211915 5405 leader_election.cc:304] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c8e8afd3f5411490f56ebf57aee109, 6f7f47d877a8407185013dba2959fddd, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:48.212271 11456 raft_consensus.cc:2798] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:48.212348 11456 raft_consensus.cc:695] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Becoming Leader. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Running, Role: LEADER
I20250112 21:06:48.212402 11456 consensus_queue.cc:237] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.212976 3665 catalog_manager.cc:5526] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd reported cstate change: term changed from 1 to 2, leader changed from fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) to 6f7f47d877a8407185013dba2959fddd (127.26.82.69). New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.215265 11536 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:48.215474 11536 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:48.215588 11536 ts_tablet_manager.cc:1397] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.002s
I20250112 21:06:48.215742 11536 raft_consensus.cc:357] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:48.215816 11536 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:48.215961 11536 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:48.216194 11536 ts_tablet_manager.cc:1428] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250112 21:06:48.216361 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-2e6bf2e45872401f8245a586b6fe7431 received from {username='slave'} at 127.26.82.69:53179
I20250112 21:06:48.216408 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-2e6bf2e45872401f8245a586b6fe7431 on tablet 2e6bf2e45872401f8245a586b6fe7431 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:48.219260 11541 ts_tablet_manager.cc:927] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:48.219393 11541 tablet_copy_client.cc:323] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:48.219645 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 3c95203cf74b4b90867a9a3a334dbb21 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:58797)
I20250112 21:06:48.219718 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 3c95203cf74b4b90867a9a3a334dbb21 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:58797: session id = 38c8e8afd3f5411490f56ebf57aee109-3c95203cf74b4b90867a9a3a334dbb21
I20250112 21:06:48.220222 3958 tablet_copy_source_session.cc:215] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:48.221052 11541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c95203cf74b4b90867a9a3a334dbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:48.222379 11541 tablet_copy_client.cc:806] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:48.222515 11541 tablet_copy_client.cc:670] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:48.223675 11541 tablet_copy_client.cc:538] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:48.224424 11541 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:48.225261 4199 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e"
dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46430
I20250112 21:06:48.225334 4199 raft_consensus.cc:604] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:48.229424 11541 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:48.229606 11541 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:48.229705 11541 ts_tablet_manager.cc:1397] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250112 21:06:48.229804 11541 raft_consensus.cc:357] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.229856 11541 raft_consensus.cc:738] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:48.229898 11541 consensus_queue.cc:260] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.229991 11541 ts_tablet_manager.cc:1428] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:48.231115 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-3c95203cf74b4b90867a9a3a334dbb21 received from {username='slave'} at 127.26.82.72:58797
I20250112 21:06:48.231170 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-3c95203cf74b4b90867a9a3a334dbb21 on tablet 3c95203cf74b4b90867a9a3a334dbb21 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:48.244915 4067 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.245466 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.245661 9682 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.246464 9682 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.246862 5864 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.247674 3665 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.249365 4067 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.249711 3916 tablet_service.cc:1514] Processing DeleteTablet for tablet 102b9dd051ff4a2a8da8ff4692e56256 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:48.249801 11043 tablet_replica.cc:331] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:48.249853 11043 raft_consensus.cc:2238] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:48.249943 11043 raft_consensus.cc:2267] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 21:06:48.251684 4002 consensus_peers.cc:487] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 102b9dd051ff4a2a8da8ff4692e56256. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.251704 11043 ts_tablet_manager.cc:1905] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:48.253173 11043 ts_tablet_manager.cc:1918] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:48.253232 11043 log.cc:1198] T 102b9dd051ff4a2a8da8ff4692e56256 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:48.253662 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 102b9dd051ff4a2a8da8ff4692e56256 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:48.254421 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:48.254666 10642 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.255411 10642 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.255509 5864 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.256203 3649 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 102b9dd051ff4a2a8da8ff4692e56256 with cas_config_opid_index 6: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:48.256501 3664 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.266536 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.269474 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.273486 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.282524 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.283048 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.284466 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.291523 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.292270 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.308719 4199 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 21:06:48.308924 9682 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:48.314190 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.317628 4199 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e"
dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46464
I20250112 21:06:48.317706 4199 raft_consensus.cc:604] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:48.319829 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.325047 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.329126 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.329638 5864 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 21:06:48.329918 9682 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:48.330708 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.335419 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.337579 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.339468 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.345211 3806 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 21:06:48.346474 11652 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 21:06:48.347798 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.351246 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.356292 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.356611 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.365948 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.366319 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.367961 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.371222 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.377795 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.386185 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.387423 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.389362 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.389469 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.401134 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.401322 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.416121 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.419994 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.426798 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.427920 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.429291 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.430827 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.435624 5844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.435985 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.438318 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.442632 6680 raft_consensus.cc:988] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: : Instructing follower 38c8e8afd3f5411490f56ebf57aee109 to start an election
I20250112 21:06:48.442690 6680 raft_consensus.cc:1076] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Signalling peer 38c8e8afd3f5411490f56ebf57aee109 to start an election
I20250112 21:06:48.442894 5864 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e"
dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
from {username='slave'} at 127.26.82.68:49499
I20250112 21:06:48.443009 5864 raft_consensus.cc:491] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:48.443023 4178 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.443042 5864 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.443825 5864 raft_consensus.cc:513] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.444123 5864 leader_election.cc:290] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [CANDIDATE]: Term 2 election: Requested vote from peers 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:48.444885 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "38c8e8afd3f5411490f56ebf57aee109" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:48.444993 4199 raft_consensus.cc:3049] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:48.445014 4199 raft_consensus.cc:738] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:48.445052 4199 consensus_queue.cc:260] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.445130 4199 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.445797 4199 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c8e8afd3f5411490f56ebf57aee109 in term 2.
I20250112 21:06:48.446300 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "38c8e8afd3f5411490f56ebf57aee109" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:48.446375 3806 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.446738 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.447111 3806 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c8e8afd3f5411490f56ebf57aee109 in term 2.
I20250112 21:06:48.447268 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "38c8e8afd3f5411490f56ebf57aee109" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:48.447340 3936 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.447964 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.448046 3936 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c8e8afd3f5411490f56ebf57aee109 in term 2.
I20250112 21:06:48.448412 5798 leader_election.cc:304] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 38c8e8afd3f5411490f56ebf57aee109, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:48.448520 7648 raft_consensus.cc:2798] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:48.448604 7648 raft_consensus.cc:695] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 LEADER]: Becoming Leader. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Running, Role: LEADER
I20250112 21:06:48.448699 7648 consensus_queue.cc:237] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.449440 3665 catalog_manager.cc:5526] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: term changed from 1 to 2, leader changed from 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) to 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72). New cstate: current_term: 2 leader_uuid: "38c8e8afd3f5411490f56ebf57aee109" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.452162 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.456249 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.458884 5602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc2e530f44745948c4607c5f81b1d0e" candidate_uuid: "38c8e8afd3f5411490f56ebf57aee109" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
I20250112 21:06:48.458987 5602 raft_consensus.cc:3054] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:48.459717 5602 raft_consensus.cc:2463] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38c8e8afd3f5411490f56ebf57aee109 in term 2.
I20250112 21:06:48.460827 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.466053 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.473619 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.486831 3936 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:48.487253 5733 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.487460 9065 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.488258 9065 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.489063 5733 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.491149 3664 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 5 to 6, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.492074 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:48.492422 11988 tablet_replica.cc:331] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:48.492491 3936 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.492535 11988 raft_consensus.cc:2238] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:48.492724 11988 raft_consensus.cc:2267] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:48.493079 11988 ts_tablet_manager.cc:1905] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20250112 21:06:48.493268 3872 consensus_peers.cc:487] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d2ae40fed4b41218aa58c523d7d9785. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.494102 5733 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:48.494225 4199 consensus_queue.cc:237] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.494272 8535 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.494717 5470 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.494796 4067 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.495074 6680 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20250112 21:06:48.495338 4133 consensus_peers.cc:487] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 979cf479af044430904e610adee50359. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.495398 6680 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.495999 9065 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.496708 11988 ts_tablet_manager.cc:1918] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:48.496769 11988 log.cc:1198] T 0d2ae40fed4b41218aa58c523d7d9785 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:48.497035 3650 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 0d2ae40fed4b41218aa58c523d7d9785 with cas_config_opid_index 6: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:48.497195 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 0d2ae40fed4b41218aa58c523d7d9785 (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:48.497167 3665 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 6 to 7, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.497478 6680 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.497478 5733 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.498709 4067 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.499485 5470 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.499577 3665 catalog_manager.cc:5526] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 5 to 6, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.510911 3807 consensus_queue.cc:237] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.511322 5864 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.511511 11823 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.512274 11823 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.512748 4199 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:48.513010 11456 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.513906 3656 catalog_manager.cc:5526] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.514053 5864 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.515077 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9176ff75364d81a5b3bdd0e7e10673 with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:48.515100 3807 consensus_queue.cc:237] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.515180 10755 tablet_replica.cc:331] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:48.515244 10755 raft_consensus.cc:2238] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:48.515295 10755 raft_consensus.cc:2267] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:48.515534 10755 ts_tablet_manager.cc:1905] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:48.515882 5864 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:48.516873 10755 ts_tablet_manager.cc:1918] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:48.516937 10755 log.cc:1198] T dd9176ff75364d81a5b3bdd0e7e10673 P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/dd9176ff75364d81a5b3bdd0e7e10673
I20250112 21:06:48.517264 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet dd9176ff75364d81a5b3bdd0e7e10673 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
W20250112 21:06:48.517325 3740 consensus_peers.cc:487] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd9176ff75364d81a5b3bdd0e7e10673. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.518764 12039 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.519641 11056 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.519845 5864 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.520741 3650 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet dd9176ff75364d81a5b3bdd0e7e10673 with cas_config_opid_index 6: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:48.521093 3663 catalog_manager.cc:5526] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.531200 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.533622 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.538043 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.538132 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.538530 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.549814 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.555655 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.561182 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:48.561509 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.561455 11456 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.569774 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:48.570196 11456 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.580297 3936 consensus_queue.cc:237] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:48.580984 4199 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.581193 9065 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.581343 5470 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.581511 9065 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20250112 21:06:48.582073 3873 consensus_peers.cc:487] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3fae2c2254d04e7a819f5d8638fd97ea. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.582352 9065 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.583582 4199 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.583673 3662 catalog_manager.cc:5526] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.585564 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.586642 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.588034 5470 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.592005 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.596887 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.601826 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.604312 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.609126 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.618791 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.624953 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.625536 5864 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:48.625757 11456 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.627076 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.633756 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.636936 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.640754 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.646320 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.646461 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.647706 3916 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.647789 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.652213 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.653123 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.658483 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.661968 12223 ts_tablet_manager.cc:927] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:48.662092 12223 tablet_copy_client.cc:323] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:48.662220 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.662312 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.662997 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:37341)
I20250112 21:06:48.663053 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:37341: session id = 67a5a7038ced40ddb6f45fdbfd99825c-102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:48.663595 4089 tablet_copy_source_session.cc:215] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:48.664361 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.668108 5733 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:48.668524 5470 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:48.668565 4199 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:48.668821 9434 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:48.668951 9434 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:48.670471 9434 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:48.670563 4199 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:48.671682 5470 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:48.672778 3665 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:48.674032 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 345f4bf45767475ca481525f3d26a34a with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:48.674144 11988 tablet_replica.cc:331] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:48.674197 11988 raft_consensus.cc:2238] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:48.674252 11988 raft_consensus.cc:2267] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:48.674502 11988 ts_tablet_manager.cc:1905] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:48.675436 12223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b9dd051ff4a2a8da8ff4692e56256. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:48.675909 11988 ts_tablet_manager.cc:1918] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:48.676016 11988 log.cc:1198] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/345f4bf45767475ca481525f3d26a34a
I20250112 21:06:48.677229 12223 tablet_copy_client.cc:806] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:48.677316 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 345f4bf45767475ca481525f3d26a34a (table rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]) successfully deleted
I20250112 21:06:48.677407 12223 tablet_copy_client.cc:670] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:48.678200 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.678484 12223 tablet_copy_client.cc:538] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:48.679205 12223 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:48.683908 12223 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:48.684121 12223 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:48.684221 12223 ts_tablet_manager.cc:1397] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250112 21:06:48.684367 12223 raft_consensus.cc:357] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.684430 12223 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:48.684470 12223 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.684605 12223 ts_tablet_manager.cc:1428] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.003s sys 0.000s
I20250112 21:06:48.684862 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-102b9dd051ff4a2a8da8ff4692e56256 received from {username='slave'} at 127.26.82.70:37341
I20250112 21:06:48.684921 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-102b9dd051ff4a2a8da8ff4692e56256 on tablet 102b9dd051ff4a2a8da8ff4692e56256 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:48.688995 3807 consensus_queue.cc:237] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:48.689669 5602 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.689728 4067 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20250112 21:06:48.689944 3742 consensus_peers.cc:487] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47ca3308b60c4367ab34f25ed5ecb0c0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.690052 11056 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.690531 11056 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.691547 11823 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.692919 3807 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:48.693091 4199 consensus_queue.cc:237] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:48.693696 5733 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.693897 6680 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.694061 11456 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.694595 6680 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.694701 5602 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.695636 4067 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.695713 3663 catalog_manager.cc:5526] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 5 to 6, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.695782 3656 catalog_manager.cc:5526] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.697252 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ff25b3286e45af8ddd38965062fca5 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:48.697360 11043 tablet_replica.cc:331] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:48.697433 11043 raft_consensus.cc:2238] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:48.697474 11043 raft_consensus.cc:2267] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:48.697788 11043 ts_tablet_manager.cc:1905] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:48.699039 3936 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:48.699296 7597 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.699430 11043 ts_tablet_manager.cc:1918] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20250112 21:06:48.699493 11043 log.cc:1198] T 74ff25b3286e45af8ddd38965062fca5 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/74ff25b3286e45af8ddd38965062fca5
I20250112 21:06:48.699934 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 74ff25b3286e45af8ddd38965062fca5 (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:48.700397 4199 consensus_queue.cc:237] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
W20250112 21:06:48.701048 4133 consensus_peers.cc:487] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 74ff25b3286e45af8ddd38965062fca5. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.701092 5733 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:48.702593 5732 raft_consensus.cc:1270] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:48.703086 10518 consensus_queue.cc:1035] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:48.703342 5470 raft_consensus.cc:1212] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20250112 21:06:48.703840 10518 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.704031 5732 raft_consensus.cc:2949] T 74ff25b3286e45af8ddd38965062fca5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.704911 3649 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 74ff25b3286e45af8ddd38965062fca5 with cas_config_opid_index 6: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:48.705963 3656 catalog_manager.cc:5526] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:48.718230 3936 consensus_queue.cc:237] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:48.718801 5470 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:48.718989 12153 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:48.719189 5732 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20250112 21:06:48.719493 3871 consensus_peers.cc:487] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): Couldn't send request to peer 2565f97e9cc54700ab2ce39aa7a546f8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.719594 12153 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:48.720201 10717 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:48.720752 5732 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:48.721781 5470 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:48.721830 3664 catalog_manager.cc:5526] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:48.732724 5864 raft_consensus.cc:1212] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20250112 21:06:48.757428 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.757974 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.764164 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.767082 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.772637 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.773582 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.777369 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.777396 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.780164 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.787787 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.791035 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.792846 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.794548 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.795769 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.801612 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.805672 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.806269 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.809091 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.817190 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.818174 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.820356 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.824255 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.828241 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.831947 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.835896 3936 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21"
dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54930
I20250112 21:06:48.835980 3936 raft_consensus.cc:604] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:48.856164 3807 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:48.856673 7648 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.861510 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.863006 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.868724 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.871759 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.872148 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.872185 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.875247 4067 consensus_queue.cc:237] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.875890 3807 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:48.876343 11652 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.876883 5864 raft_consensus.cc:1270] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:48.876888 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.877228 11652 consensus_queue.cc:1035] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.877337 11652 raft_consensus.cc:2949] T 5da4fd48e1f24680babd4f1874b31b5f P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.877344 3807 raft_consensus.cc:2949] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.878576 5864 raft_consensus.cc:2949] T 5da4fd48e1f24680babd4f1874b31b5f P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.879622 12223 ts_tablet_manager.cc:927] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:48.879745 12223 tablet_copy_client.cc:323] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:48.881151 3663 catalog_manager.cc:5526] T 5da4fd48e1f24680babd4f1874b31b5f P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 4 to 6, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:48.881235 3936 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:48.881319 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 979cf479af044430904e610adee50359 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:48269)
I20250112 21:06:48.881373 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 979cf479af044430904e610adee50359 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:48269: session id = 67a5a7038ced40ddb6f45fdbfd99825c-979cf479af044430904e610adee50359
I20250112 21:06:48.881911 4220 tablet_copy_source_session.cc:215] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:48.882036 11919 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.882560 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 5da4fd48e1f24680babd4f1874b31b5f with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:48.882690 10755 tablet_replica.cc:331] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:48.882773 10755 raft_consensus.cc:2238] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:48.882803 10755 raft_consensus.cc:2267] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:48.883109 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.883126 10755 ts_tablet_manager.cc:1905] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:48.884497 10755 ts_tablet_manager.cc:1918] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20250112 21:06:48.884542 10755 log.cc:1198] T 5da4fd48e1f24680babd4f1874b31b5f P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/5da4fd48e1f24680babd4f1874b31b5f
I20250112 21:06:48.884923 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 5da4fd48e1f24680babd4f1874b31b5f (table rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]) successfully deleted
I20250112 21:06:48.884955 12223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979cf479af044430904e610adee50359. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:48.889114 12223 tablet_copy_client.cc:806] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:48.889218 12223 tablet_copy_client.cc:670] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:48.892416 5602 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:48.892449 4199 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250112 21:06:48.892622 11919 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.893322 12540 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.896818 12223 tablet_copy_client.cc:538] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:48.897976 12223 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:48.900002 12568 ts_tablet_manager.cc:927] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:48.900141 12568 tablet_copy_client.cc:323] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:48.900372 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:58797)
I20250112 21:06:48.900425 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:58797: session id = 38c8e8afd3f5411490f56ebf57aee109-0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:48.901163 3958 tablet_copy_source_session.cc:215] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:48.903471 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:48.904721 12568 tablet_copy_client.cc:806] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:48.904820 12568 tablet_copy_client.cc:670] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:48.905208 12223 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:48.905581 12223 tablet_bootstrap.cc:492] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:48.905689 12223 ts_tablet_manager.cc:1397] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20250112 21:06:48.905827 12223 raft_consensus.cc:357] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.905907 12223 raft_consensus.cc:738] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:48.905988 12223 consensus_queue.cc:260] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.906100 12223 ts_tablet_manager.cc:1428] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:48.906841 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-979cf479af044430904e610adee50359 received from {username='slave'} at 127.26.82.70:48269
I20250112 21:06:48.906891 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-979cf479af044430904e610adee50359 on tablet 979cf479af044430904e610adee50359 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:48.908267 12568 tablet_copy_client.cc:538] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:48.908892 12568 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:48.913115 12223 ts_tablet_manager.cc:927] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:48.913229 12223 tablet_copy_client.cc:323] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:48.913380 12568 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:48.913401 3827 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet dd9176ff75364d81a5b3bdd0e7e10673 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:50395)
I20250112 21:06:48.913462 3827 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet dd9176ff75364d81a5b3bdd0e7e10673 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:50395: session id = 67a5a7038ced40ddb6f45fdbfd99825c-dd9176ff75364d81a5b3bdd0e7e10673
I20250112 21:06:48.913583 12568 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:48.913686 12568 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20250112 21:06:48.913810 12568 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.913882 12568 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:48.913949 12568 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:48.914098 12568 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:48.914113 3827 tablet_copy_source_session.cc:215] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:48.914299 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-0d2ae40fed4b41218aa58c523d7d9785 received from {username='slave'} at 127.26.82.72:58797
I20250112 21:06:48.914348 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-0d2ae40fed4b41218aa58c523d7d9785 on tablet 0d2ae40fed4b41218aa58c523d7d9785 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:48.916911 12223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd9176ff75364d81a5b3bdd0e7e10673. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:48.918104 12223 tablet_copy_client.cc:806] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:48.918200 12223 tablet_copy_client.cc:670] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:48.924332 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.926391 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.926421 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.927430 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.935540 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.936019 12223 tablet_copy_client.cc:538] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:48.936829 12223 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:48.937089 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:48.940861 12223 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:48.941066 12223 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:48.941149 12223 ts_tablet_manager.cc:1397] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20250112 21:06:48.941254 12223 raft_consensus.cc:357] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.941309 12223 raft_consensus.cc:738] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:48.941358 12223 consensus_queue.cc:260] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:48.941440 12223 ts_tablet_manager.cc:1428] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:48.941591 3827 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-dd9176ff75364d81a5b3bdd0e7e10673 received from {username='slave'} at 127.26.82.70:50395
I20250112 21:06:48.941653 3827 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-dd9176ff75364d81a5b3bdd0e7e10673 on tablet dd9176ff75364d81a5b3bdd0e7e10673 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:48.942257 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.943429 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:48.977677 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:48.978873 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:48.980158 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:48.985795 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:48.987671 4199 consensus_queue.cc:237] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:48.992810 4067 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.994285 5602 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20250112 21:06:48.994388 11997 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 21:06:48.994606 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:48.995103 11997 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20250112 21:06:48.995510 4134 consensus_peers.cc:487] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d -> Peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): Couldn't send request to peer fd8fa04957924e9f83afd61f9adfb455. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:48.996547 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:49.003460 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.009826 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:49.016067 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.019140 11997 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:49.019548 5602 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:49.020082 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.021335 3659 catalog_manager.cc:5526] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:49.021492 4067 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:49.022722 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.023787 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.025805 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.028856 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:49.029106 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:49.030041 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:49.034538 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.039966 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.040277 5602 raft_consensus.cc:1212] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20250112 21:06:49.041798 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:49.045270 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.046078 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:49.046299 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:49.056910 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:49.065590 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:49.077783 5864 consensus_queue.cc:237] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.078855 3807 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.079051 3936 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.079548 11919 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.081594 5602 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.082242 12540 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.082325 11919 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.082566 4049 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:49.082828 3938 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431"
dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55004
I20250112 21:06:49.082888 3938 raft_consensus.cc:604] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:49.083424 12540 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.083527 5602 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.084007 3936 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.084844 3665 catalog_manager.cc:5526] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 5 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 2 leader_uuid: "38c8e8afd3f5411490f56ebf57aee109" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.085461 3807 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.085981 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc2e530f44745948c4607c5f81b1d0e with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:49.086086 10755 tablet_replica.cc:331] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:49.086166 10755 raft_consensus.cc:2238] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.086210 10755 raft_consensus.cc:2267] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.086890 10755 ts_tablet_manager.cc:1905] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.088436 10755 ts_tablet_manager.cc:1918] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20250112 21:06:49.088481 10755 log.cc:1198] T 7fc2e530f44745948c4607c5f81b1d0e P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/7fc2e530f44745948c4607c5f81b1d0e
I20250112 21:06:49.089008 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 7fc2e530f44745948c4607c5f81b1d0e (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:49.089532 5582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250112 21:06:49.092093 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.097414 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.102808 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:49.103654 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.104024 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.104138 12153 raft_consensus.cc:988] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: : Instructing follower 2565f97e9cc54700ab2ce39aa7a546f8 to start an election
I20250112 21:06:49.104189 12153 raft_consensus.cc:1076] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Signalling peer 2565f97e9cc54700ab2ce39aa7a546f8 to start an election
I20250112 21:06:49.104353 4067 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21"
dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
from {username='slave'} at 127.26.82.66:38619
I20250112 21:06:49.104418 4067 raft_consensus.cc:491] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:49.104447 4067 raft_consensus.cc:3054] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.105214 4067 raft_consensus.cc:513] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:49.105360 4067 leader_election.cc:290] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 2 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889), 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:49.105616 5733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
I20250112 21:06:49.105710 5733 raft_consensus.cc:3054] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.106372 5733 raft_consensus.cc:2463] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:49.106563 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
I20250112 21:06:49.106626 5864 raft_consensus.cc:3054] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.107280 5864 raft_consensus.cc:2463] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:49.107285 3917 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20250112 21:06:49.107436 4003 leader_election.cc:304] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, 38c8e8afd3f5411490f56ebf57aee109, 85a6e2a0c3b94dea8a094d396cfcac4b; no voters:
I20250112 21:06:49.107528 12525 raft_consensus.cc:2798] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:49.107568 12525 raft_consensus.cc:695] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Becoming Leader. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:49.107610 12525 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:49.108156 3665 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: term changed from 1 to 2, leader changed from fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) to 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67). New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.109225 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c95203cf74b4b90867a9a3a334dbb21" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:49.109292 3936 raft_consensus.cc:3049] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:49.109313 3936 raft_consensus.cc:738] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:49.109355 3936 consensus_queue.cc:260] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:49.109448 3936 raft_consensus.cc:3054] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.110155 3936 raft_consensus.cc:2463] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:49.110479 9065 raft_consensus.cc:988] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: : Instructing follower 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:49.110523 9065 raft_consensus.cc:1076] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Signalling peer 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:49.110797 5602 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431"
dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
from {username='slave'} at 127.26.82.66:37129
I20250112 21:06:49.110883 5602 raft_consensus.cc:491] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:49.110908 5602 raft_consensus.cc:3054] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.111569 5602 raft_consensus.cc:513] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:49.111840 5602 leader_election.cc:290] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 2 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:49.113137 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:49.113202 4067 raft_consensus.cc:3054] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.113788 4067 raft_consensus.cc:2463] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:49.114681 3938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:49.114758 3938 raft_consensus.cc:3049] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:49.114780 3938 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Running, Role: LEADER
I20250112 21:06:49.114820 3938 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:49.114887 3938 raft_consensus.cc:3054] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.115526 3938 raft_consensus.cc:2463] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:49.115847 12824 ts_tablet_manager.cc:927] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:49.115850 5536 leader_election.cc:304] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, 67a5a7038ced40ddb6f45fdbfd99825c, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:49.115970 12553 raft_consensus.cc:2798] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:49.116043 12553 raft_consensus.cc:695] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Becoming Leader. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Running, Role: LEADER
I20250112 21:06:49.116092 12824 tablet_copy_client.cc:323] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:49.116111 12553 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:49.116868 3827 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet 47ca3308b60c4367ab34f25ed5ecb0c0 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:34353)
I20250112 21:06:49.116945 3827 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet 47ca3308b60c4367ab34f25ed5ecb0c0 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:34353: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-47ca3308b60c4367ab34f25ed5ecb0c0
I20250112 21:06:49.117475 3827 tablet_copy_source_session.cc:215] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.117642 3665 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: term changed from 1 to 2, leader changed from fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) to 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70). New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.117663 5470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6bf2e45872401f8245a586b6fe7431" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "6f7f47d877a8407185013dba2959fddd"
I20250112 21:06:49.117733 5470 raft_consensus.cc:3054] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.118396 12824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47ca3308b60c4367ab34f25ed5ecb0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.118466 5470 raft_consensus.cc:2463] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:49.119858 12824 tablet_copy_client.cc:806] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.119964 12824 tablet_copy_client.cc:670] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.120929 12824 tablet_copy_client.cc:538] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.121585 12824 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:49.126005 12824 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.126641 12824 tablet_bootstrap.cc:492] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:49.126830 12824 ts_tablet_manager.cc:1397] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20250112 21:06:49.127022 12824 raft_consensus.cc:357] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.127107 12824 raft_consensus.cc:738] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.127168 12824 consensus_queue.cc:260] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.127384 12824 ts_tablet_manager.cc:1428] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:49.127739 3827 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-47ca3308b60c4367ab34f25ed5ecb0c0 received from {username='slave'} at 127.26.82.71:34353
I20250112 21:06:49.127794 3827 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-47ca3308b60c4367ab34f25ed5ecb0c0 on tablet 47ca3308b60c4367ab34f25ed5ecb0c0 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:49.128134 5470 consensus_queue.cc:237] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:49.128571 3807 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.128600 5864 raft_consensus.cc:1270] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.128922 11456 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.129058 12115 consensus_queue.cc:1035] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.130263 11456 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.131520 3665 catalog_manager.cc:5526] T 0fce4710d0784d02991a6465522323d1 P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.131764 5864 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.132875 3807 raft_consensus.cc:2949] T 0fce4710d0784d02991a6465522323d1 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.133877 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fce4710d0784d02991a6465522323d1 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:49.134028 11043 tablet_replica.cc:331] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:49.134094 11043 raft_consensus.cc:2238] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.134125 11043 raft_consensus.cc:2267] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.134799 11043 ts_tablet_manager.cc:1905] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.135416 12859 ts_tablet_manager.cc:927] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:49.135762 12859 tablet_copy_client.cc:323] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:49.136636 11043 ts_tablet_manager.cc:1918] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:49.136747 11043 log.cc:1198] T 0fce4710d0784d02991a6465522323d1 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/0fce4710d0784d02991a6465522323d1
I20250112 21:06:49.137193 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 0fce4710d0784d02991a6465522323d1 (table rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]) successfully deleted
I20250112 21:06:49.137257 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet c93763a09f7d4460b1874bf328fb06f5 from peer 2565f97e9cc54700ab2ce39aa7a546f8 ({username='slave'} at 127.26.82.67:54355)
I20250112 21:06:49.137305 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet c93763a09f7d4460b1874bf328fb06f5 from peer 2565f97e9cc54700ab2ce39aa7a546f8 at {username='slave'} at 127.26.82.67:54355: session id = 2565f97e9cc54700ab2ce39aa7a546f8-c93763a09f7d4460b1874bf328fb06f5
I20250112 21:06:49.137849 3958 tablet_copy_source_session.cc:215] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.138288 3807 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355"
dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37814
I20250112 21:06:49.138625 3807 raft_consensus.cc:604] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:49.138844 12859 ts_tablet_manager.cc:1905] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:49.140566 12859 ts_tablet_manager.cc:1918] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20250112 21:06:49.140784 12859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c93763a09f7d4460b1874bf328fb06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.141911 12859 tablet_copy_client.cc:806] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.142024 12859 tablet_copy_client.cc:670] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.142813 12859 tablet_copy_client.cc:538] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.143513 12859 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:49.148777 12859 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.148979 12859 tablet_bootstrap.cc:492] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap complete.
I20250112 21:06:49.149070 12859 ts_tablet_manager.cc:1397] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20250112 21:06:49.149183 12859 raft_consensus.cc:357] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.149255 12859 raft_consensus.cc:738] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.149322 12859 consensus_queue.cc:260] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.149583 12859 ts_tablet_manager.cc:1428] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:49.149775 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-c93763a09f7d4460b1874bf328fb06f5 received from {username='slave'} at 127.26.82.67:54355
I20250112 21:06:49.149820 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-c93763a09f7d4460b1874bf328fb06f5 on tablet c93763a09f7d4460b1874bf328fb06f5 with peer 2565f97e9cc54700ab2ce39aa7a546f8
I20250112 21:06:49.154461 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.159144 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.161063 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:49.165278 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.171043 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.171095 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.172024 5582 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.181361 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.181411 12824 ts_tablet_manager.cc:927] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:49.185324 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.186425 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.189275 12824 tablet_copy_client.cc:323] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:49.189471 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 3fae2c2254d04e7a819f5d8638fd97ea from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49463)
I20250112 21:06:49.189522 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 3fae2c2254d04e7a819f5d8638fd97ea from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49463: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-3fae2c2254d04e7a819f5d8638fd97ea
I20250112 21:06:49.190004 3958 tablet_copy_source_session.cc:215] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.190778 12824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fae2c2254d04e7a819f5d8638fd97ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.192277 12824 tablet_copy_client.cc:806] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.192384 12824 tablet_copy_client.cc:670] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.193531 12824 tablet_copy_client.cc:538] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.194271 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.194307 12824 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:49.197369 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.200960 12824 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.201169 12824 tablet_bootstrap.cc:492] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:49.201454 12824 ts_tablet_manager.cc:1397] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20250112 21:06:49.201625 12824 raft_consensus.cc:357] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.201761 12824 raft_consensus.cc:738] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.201856 12824 consensus_queue.cc:260] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.202152 12824 ts_tablet_manager.cc:1428] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:49.202351 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-3fae2c2254d04e7a819f5d8638fd97ea received from {username='slave'} at 127.26.82.71:49463
I20250112 21:06:49.202392 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-3fae2c2254d04e7a819f5d8638fd97ea on tablet 3fae2c2254d04e7a819f5d8638fd97ea with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:49.203584 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.209605 5582 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.213387 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.217433 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:49.221844 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.223291 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.224836 3787 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:49.234927 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.235100 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.235216 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.235846 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.237177 5582 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.246207 11056 raft_consensus.cc:988] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: : Instructing follower 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:49.246266 11056 raft_consensus.cc:1076] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Signalling peer 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:49.246518 5602 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355"
dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
from {username='slave'} at 127.26.82.65:54203
I20250112 21:06:49.246594 5602 raft_consensus.cc:491] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:49.246619 5602 raft_consensus.cc:3054] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.247450 5602 raft_consensus.cc:513] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.247663 5602 leader_election.cc:290] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 2 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569), 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:49.247773 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:49.247848 3807 raft_consensus.cc:3049] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:49.247876 3807 raft_consensus.cc:738] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:49.247922 3807 consensus_queue.cc:260] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.248010 3807 raft_consensus.cc:3054] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.249313 3807 raft_consensus.cc:2463] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:49.249451 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:49.249526 4199 raft_consensus.cc:3054] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.250330 4199 raft_consensus.cc:2463] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:49.250623 5536 leader_election.cc:304] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 67a5a7038ced40ddb6f45fdbfd99825c, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:49.250810 12243 raft_consensus.cc:2798] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:49.250859 12243 raft_consensus.cc:695] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Becoming Leader. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Running, Role: LEADER
I20250112 21:06:49.250909 12243 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.251976 3664 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: term changed from 1 to 2, leader changed from bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) to 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70). New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.252228 5732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b6276ae83948b6b4ea2771d9ce8355" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
I20250112 21:06:49.252302 5732 raft_consensus.cc:3054] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.253074 5732 raft_consensus.cc:2463] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:49.263970 5470 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:49.264508 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.264823 11456 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.264896 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.265095 11456 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.265910 7597 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.266569 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.267693 5864 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.267771 3664 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 6 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.269484 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:49.269582 10755 tablet_replica.cc:331] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:49.269949 10755 raft_consensus.cc:2238] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.270025 10755 raft_consensus.cc:2267] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.270408 10755 ts_tablet_manager.cc:1905] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.271910 10755 ts_tablet_manager.cc:1918] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:49.271987 10755 log.cc:1198] T de6b6ac6eeff431b9ac8cce47ea3a59e P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:49.272344 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet de6b6ac6eeff431b9ac8cce47ea3a59e (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:49.292913 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.297178 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.297958 12223 ts_tablet_manager.cc:927] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:49.298128 12223 tablet_copy_client.cc:323] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:49.298321 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 74ff25b3286e45af8ddd38965062fca5 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:48269)
I20250112 21:06:49.298398 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 74ff25b3286e45af8ddd38965062fca5 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:48269: session id = 67a5a7038ced40ddb6f45fdbfd99825c-74ff25b3286e45af8ddd38965062fca5
I20250112 21:06:49.298874 4220 tablet_copy_source_session.cc:215] T 74ff25b3286e45af8ddd38965062fca5 P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.299328 12223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ff25b3286e45af8ddd38965062fca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.300551 12223 tablet_copy_client.cc:806] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.300673 12223 tablet_copy_client.cc:670] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.302742 12223 tablet_copy_client.cc:538] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.303434 12223 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:49.308349 12223 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.310060 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.311440 12223 tablet_bootstrap.cc:492] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:49.311587 12223 ts_tablet_manager.cc:1397] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20250112 21:06:49.311748 12223 raft_consensus.cc:357] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.311882 12223 raft_consensus.cc:738] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.311992 12223 consensus_queue.cc:260] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.312176 12223 ts_tablet_manager.cc:1428] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:49.312506 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-74ff25b3286e45af8ddd38965062fca5 received from {username='slave'} at 127.26.82.70:48269
I20250112 21:06:49.312553 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-74ff25b3286e45af8ddd38965062fca5 on tablet 74ff25b3286e45af8ddd38965062fca5 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:49.317046 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.317463 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.320794 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.323513 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.331509 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.345408 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.350917 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.351795 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.353345 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.353498 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.365679 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.370491 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.372416 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.373261 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.375362 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.376798 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.383325 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.383646 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.384783 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.385138 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.390875 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.397327 5864 raft_consensus.cc:1212] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20250112 21:06:49.412569 5470 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.413304 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
W20250112 21:06:49.413487 5404 consensus_peers.cc:487] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd -> Peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): Couldn't send request to peer 2565f97e9cc54700ab2ce39aa7a546f8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:49.413578 12115 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:49.413847 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:49.414364 12115 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:49.415185 12115 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.415310 5864 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.416692 3656 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 8 to 9, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.416983 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.424934 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.429014 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.432947 5864 consensus_queue.cc:237] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.433332 3938 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.433362 5602 raft_consensus.cc:1270] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 38c8e8afd3f5411490f56ebf57aee109: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.433526 11919 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.433658 11919 consensus_queue.cc:1035] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.434466 12540 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.434620 3938 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.436053 3663 catalog_manager.cc:5526] T 7fc2e530f44745948c4607c5f81b1d0e P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 2 leader_uuid: "38c8e8afd3f5411490f56ebf57aee109" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.436292 5602 raft_consensus.cc:2949] T 7fc2e530f44745948c4607c5f81b1d0e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.436524 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.436892 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc2e530f44745948c4607c5f81b1d0e with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:49.437191 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.437367 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.437638 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.442484 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.444284 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.452891 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.456660 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.460762 13276 tablet_replica.cc:331] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:49.460883 13276 raft_consensus.cc:2238] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.461030 13276 raft_consensus.cc:2267] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.463788 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.463948 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.464227 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.466917 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.471149 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.477957 13276 ts_tablet_manager.cc:1905] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.478514 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.478825 5602 raft_consensus.cc:1212] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20250112 21:06:49.480602 4067 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:49.481142 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:49.481153 5602 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:49.481329 12525 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20250112 21:06:49.481407 4002 consensus_peers.cc:487] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): Couldn't send request to peer 1d1923072e1c45778fa5afdcfda8144d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:49.481446 12525 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.483304 13276 ts_tablet_manager.cc:1918] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:49.483371 13276 log.cc:1198] T 7fc2e530f44745948c4607c5f81b1d0e P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/7fc2e530f44745948c4607c5f81b1d0e
I20250112 21:06:49.483392 12524 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:49.483573 5864 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:49.484032 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 7fc2e530f44745948c4607c5f81b1d0e (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:49.484898 3664 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:49.486474 5602 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:49.506522 3938 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.506852 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.507957 5602 raft_consensus.cc:1212] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20250112 21:06:49.515017 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.526953 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.527655 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.531944 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.532032 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.533026 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.536880 3938 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.537477 5732 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:49.537657 9065 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20250112 21:06:49.537878 3871 consensus_peers.cc:487] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): Couldn't send request to peer 2565f97e9cc54700ab2ce39aa7a546f8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:49.538081 5864 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:49.538281 9065 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.539285 9065 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.539383 5864 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.540642 3659 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.540714 5732 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:49.543252 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.544423 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.549562 4067 raft_consensus.cc:1212] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20250112 21:06:49.576400 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.579135 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.579252 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.581039 5732 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.581058 5864 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.581221 12524 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.581339 12524 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.586208 3938 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.586386 12524 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.586573 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.588471 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.595372 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.595932 13518 ts_tablet_manager.cc:927] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:49.596118 13518 tablet_copy_client.cc:323] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:49.596357 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.596490 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet dd01159cf53642739a8a1dbcacbc933d from peer fd8fa04957924e9f83afd61f9adfb455 ({username='slave'} at 127.26.82.66:57339)
I20250112 21:06:49.596558 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet dd01159cf53642739a8a1dbcacbc933d from peer fd8fa04957924e9f83afd61f9adfb455 at {username='slave'} at 127.26.82.66:57339: session id = fd8fa04957924e9f83afd61f9adfb455-dd01159cf53642739a8a1dbcacbc933d
I20250112 21:06:49.597148 4220 tablet_copy_source_session.cc:215] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.597644 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.597887 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.599308 13518 ts_tablet_manager.cc:1905] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:49.600955 13518 ts_tablet_manager.cc:1918] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.4
I20250112 21:06:49.601096 13518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd01159cf53642739a8a1dbcacbc933d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.601912 13518 tablet_copy_client.cc:806] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.602087 13518 tablet_copy_client.cc:670] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.603133 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.603942 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.604290 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.605159 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.618836 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.622084 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.626470 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.627797 13518 tablet_copy_client.cc:538] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.629142 13518 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:49.634495 13518 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.634718 13518 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Bootstrap complete.
I20250112 21:06:49.634835 13518 ts_tablet_manager.cc:1397] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250112 21:06:49.634961 13518 raft_consensus.cc:357] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:49.635053 13518 raft_consensus.cc:738] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.635151 13518 consensus_queue.cc:260] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:49.635402 13518 ts_tablet_manager.cc:1428] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250112 21:06:49.635603 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session fd8fa04957924e9f83afd61f9adfb455-dd01159cf53642739a8a1dbcacbc933d received from {username='slave'} at 127.26.82.66:57339
I20250112 21:06:49.635660 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session fd8fa04957924e9f83afd61f9adfb455-dd01159cf53642739a8a1dbcacbc933d on tablet dd01159cf53642739a8a1dbcacbc933d with peer fd8fa04957924e9f83afd61f9adfb455
I20250112 21:06:49.639966 4199 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.640179 12553 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.650082 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.651628 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.652115 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.654451 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.656483 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.657467 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.658993 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.672695 4067 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.672863 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.674408 5732 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.674954 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.675158 12553 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.675247 12243 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.680521 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.681043 3807 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:49.681973 12553 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.683997 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.684825 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.685459 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.688690 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.690196 5732 raft_consensus.cc:1212] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20250112 21:06:49.691109 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.696249 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.706307 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.707512 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.722592 5732 raft_consensus.cc:1212] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20250112 21:06:49.726230 3938 consensus_queue.cc:237] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:49.726718 4199 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:49.726718 5602 raft_consensus.cc:1270] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:49.727144 8535 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.727728 8535 consensus_queue.cc:1035] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.731596 12153 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.732991 4199 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.733127 3657 catalog_manager.cc:5526] T c16b940a9e564da89ec84b46a6cf42d7 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 6 to 7, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.734436 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.734654 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.740835 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.744382 5602 raft_consensus.cc:2949] T c16b940a9e564da89ec84b46a6cf42d7 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:49.746317 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.746588 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet c16b940a9e564da89ec84b46a6cf42d7 with delete_type TABLET_DATA_TOMBSTONED (TS 85a6e2a0c3b94dea8a094d396cfcac4b not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:56626
I20250112 21:06:49.746855 13702 tablet_replica.cc:331] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: stopping tablet replica
I20250112 21:06:49.747000 13702 raft_consensus.cc:2238] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.747121 13702 raft_consensus.cc:2267] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.747382 13702 ts_tablet_manager.cc:1905] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.749229 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.749423 13702 ts_tablet_manager.cc:1918] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:49.749534 13702 log.cc:1198] T c16b940a9e564da89ec84b46a6cf42d7 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/c16b940a9e564da89ec84b46a6cf42d7
I20250112 21:06:49.749977 3651 catalog_manager.cc:4872] TS 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): tablet c16b940a9e564da89ec84b46a6cf42d7 (table rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]) successfully deleted
I20250112 21:06:49.750528 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.750834 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.753136 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.765583 5602 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:49.766273 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.766592 12243 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.767079 3938 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.767311 12243 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.767426 12243 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.767618 5470 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.768765 3660 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 6 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.769390 3938 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.769786 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:49.771401 13721 tablet_replica.cc:331] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:49.771476 13721 raft_consensus.cc:2238] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.771601 13721 raft_consensus.cc:2267] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.775817 13721 ts_tablet_manager.cc:1905] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.777635 13721 ts_tablet_manager.cc:1918] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:49.777719 13721 log.cc:1198] T 2e6bf2e45872401f8245a586b6fe7431 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:49.779068 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 2e6bf2e45872401f8245a586b6fe7431 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:49.800313 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.801488 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.804670 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.819581 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.823997 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.823990 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.828275 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.828609 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.832037 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.833093 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.833156 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.837234 5602 raft_consensus.cc:1212] T 74ff25b3286e45af8ddd38965062fca5 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20250112 21:06:49.840523 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.847752 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.849139 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.850122 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.850358 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.850708 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.854516 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.858788 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.863126 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.865553 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.870522 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.874496 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.875598 13841 ts_tablet_manager.cc:927] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Initiating tablet copy from peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:49.875747 13841 tablet_copy_client.cc:323] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.69:40871
I20250112 21:06:49.875982 5485 tablet_copy_service.cc:140] P 6f7f47d877a8407185013dba2959fddd: Received BeginTabletCopySession request for tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 2565f97e9cc54700ab2ce39aa7a546f8 ({username='slave'} at 127.26.82.67:43195)
I20250112 21:06:49.876046 5485 tablet_copy_service.cc:161] P 6f7f47d877a8407185013dba2959fddd: Beginning new tablet copy session on tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 2565f97e9cc54700ab2ce39aa7a546f8 at {username='slave'} at 127.26.82.67:43195: session id = 2565f97e9cc54700ab2ce39aa7a546f8-de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:49.876833 5485 tablet_copy_source_session.cc:215] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.877210 13841 ts_tablet_manager.cc:1905] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:49.879590 13841 ts_tablet_manager.cc:1918] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.5
I20250112 21:06:49.879765 13841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.880640 13841 tablet_copy_client.cc:806] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.880864 13841 tablet_copy_client.cc:670] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.881650 13841 tablet_copy_client.cc:538] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.881807 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.882344 13841 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:49.885854 13841 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.886401 13841 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap complete.
I20250112 21:06:49.886549 13841 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20250112 21:06:49.886711 13841 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.886865 13841 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.886977 13841 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.887108 13841 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:49.887303 5485 tablet_copy_service.cc:342] P 6f7f47d877a8407185013dba2959fddd: Request end of tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-de6b6ac6eeff431b9ac8cce47ea3a59e received from {username='slave'} at 127.26.82.67:43195
I20250112 21:06:49.887349 5485 tablet_copy_service.cc:434] P 6f7f47d877a8407185013dba2959fddd: ending tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-de6b6ac6eeff431b9ac8cce47ea3a59e on tablet de6b6ac6eeff431b9ac8cce47ea3a59e with peer 2565f97e9cc54700ab2ce39aa7a546f8
I20250112 21:06:49.909626 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.910300 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.911257 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.913316 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.913676 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.923306 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.926316 13841 ts_tablet_manager.cc:927] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:49.926452 13841 tablet_copy_client.cc:323] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:49.926848 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 2565f97e9cc54700ab2ce39aa7a546f8 ({username='slave'} at 127.26.82.67:54355)
I20250112 21:06:49.926905 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 2565f97e9cc54700ab2ce39aa7a546f8 at {username='slave'} at 127.26.82.67:54355: session id = 2565f97e9cc54700ab2ce39aa7a546f8-0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:49.927384 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.927397 3958 tablet_copy_source_session.cc:215] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.927685 13841 ts_tablet_manager.cc:1905] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:49.929311 13841 ts_tablet_manager.cc:1918] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.4
I20250112 21:06:49.929411 13841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.930058 13841 tablet_copy_client.cc:806] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.930172 13841 tablet_copy_client.cc:670] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.930878 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.931120 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.935457 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.936058 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.936275 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.948489 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:49.951210 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.951998 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:49.952842 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:49.956894 5602 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:49.957405 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:49.957579 12243 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:49.958289 12243 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.959511 3664 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 8 to 9, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.959921 5470 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:49.960286 13841 tablet_copy_client.cc:538] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.961201 13841 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:49.961886 4067 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:49.962440 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:49.963353 5732 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.963891 13312 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.963917 5864 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:49.964165 13312 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:49.965018 13312 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.965273 13841 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.965376 13945 tablet_replica.cc:331] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:49.965437 13945 raft_consensus.cc:2238] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.965463 13841 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap complete.
I20250112 21:06:49.965493 13945 raft_consensus.cc:2267] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.965553 13841 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20250112 21:06:49.965662 13841 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.965725 13841 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.965790 13841 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:49.965907 13841 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:49.966498 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-0d2ae40fed4b41218aa58c523d7d9785 received from {username='slave'} at 127.26.82.67:54355
I20250112 21:06:49.966550 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-0d2ae40fed4b41218aa58c523d7d9785 on tablet 0d2ae40fed4b41218aa58c523d7d9785 with peer 2565f97e9cc54700ab2ce39aa7a546f8
I20250112 21:06:49.966748 5602 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
W20250112 21:06:49.969971 5536 consensus_peers.cc:487] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c -> Peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): Couldn't send request to peer fd8fa04957924e9f83afd61f9adfb455. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:49.970053 4199 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.970585 5733 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:49.970894 6680 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.971221 4067 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:49.971590 6680 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:49.971880 3664 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.971975 6680 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:49.972048 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:49.972877 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c95203cf74b4b90867a9a3a334dbb21 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:49.973134 5733 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:49.973573 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:49.972872 5732 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.974570 12553 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:49.974709 4199 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10"
dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47066
I20250112 21:06:49.974776 4199 raft_consensus.cc:604] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:49.974927 6680 raft_consensus.cc:988] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: : Instructing follower 85a6e2a0c3b94dea8a094d396cfcac4b to start an election
I20250112 21:06:49.974967 6680 raft_consensus.cc:1076] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Signalling peer 85a6e2a0c3b94dea8a094d396cfcac4b to start an election
I20250112 21:06:49.975359 3664 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:49.975811 5733 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10"
dest_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
from {username='slave'} at 127.26.82.68:47249
I20250112 21:06:49.975872 5733 raft_consensus.cc:491] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:49.975898 5733 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.976485 5733 raft_consensus.cc:513] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.976611 5733 leader_election.cc:290] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [CANDIDATE]: Term 2 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:49.976780 3658 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 9 to 10, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:49.976941 4199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
I20250112 21:06:49.977001 4199 raft_consensus.cc:3049] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:49.977028 4199 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:49.977072 4199 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.977139 4199 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.977411 3649 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 2e6bf2e45872401f8245a586b6fe7431 with cas_config_opid_index 9: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:49.977453 5582 tablet_service.cc:1514] Processing DeleteTablet for tablet 87835ddc19954987b0f17f6aa0370c10 with delete_type TABLET_DATA_TOMBSTONED (TS 67a5a7038ced40ddb6f45fdbfd99825c not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:49494
I20250112 21:06:49.977790 13945 ts_tablet_manager.cc:1905] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.977928 4199 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85a6e2a0c3b94dea8a094d396cfcac4b in term 2.
I20250112 21:06:49.978170 13959 ts_tablet_manager.cc:927] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:49.978618 13959 tablet_copy_client.cc:323] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:49.978873 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 1d1923072e1c45778fa5afdcfda8144d ({username='slave'} at 127.26.82.68:58711)
I20250112 21:06:49.978917 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 1d1923072e1c45778fa5afdcfda8144d at {username='slave'} at 127.26.82.68:58711: session id = 1d1923072e1c45778fa5afdcfda8144d-102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:49.979009 5666 leader_election.cc:304] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 85a6e2a0c3b94dea8a094d396cfcac4b; no voters:
I20250112 21:06:49.979084 12231 raft_consensus.cc:2798] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:49.979161 12231 raft_consensus.cc:695] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Becoming Leader. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Running, Role: LEADER
I20250112 21:06:49.979200 12231 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:49.979452 4089 tablet_copy_source_session.cc:215] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:49.979385 5470 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:49.979509 13945 ts_tablet_manager.cc:1918] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.8
I20250112 21:06:49.979553 13945 log.cc:1198] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:49.979542 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:49.979588 4067 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:49.979836 13959 ts_tablet_manager.cc:1905] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:49.979900 13945 tablet_replica.cc:331] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:49.979965 13945 raft_consensus.cc:2238] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.979955 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 2e6bf2e45872401f8245a586b6fe7431 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:49.980005 13945 raft_consensus.cc:2267] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.980233 4067 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85a6e2a0c3b94dea8a094d396cfcac4b in term 2.
I20250112 21:06:49.980309 13945 ts_tablet_manager.cc:1905] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.980435 5864 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:49.980782 3658 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: term changed from 1 to 2, leader changed from 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) to 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71). New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:49.982431 13959 ts_tablet_manager.cc:1918] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.4
I20250112 21:06:49.982573 13959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b9dd051ff4a2a8da8ff4692e56256. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:49.982578 13945 ts_tablet_manager.cc:1918] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:49.982640 13945 log.cc:1198] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/3c95203cf74b4b90867a9a3a334dbb21
I20250112 21:06:49.983062 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 3c95203cf74b4b90867a9a3a334dbb21 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
I20250112 21:06:49.983590 13959 tablet_copy_client.cc:806] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:49.983729 13959 tablet_copy_client.cc:670] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:49.984601 13959 tablet_copy_client.cc:538] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:49.985349 13959 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:49.985378 13965 tablet_replica.cc:331] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: stopping tablet replica
I20250112 21:06:49.985477 13965 raft_consensus.cc:2238] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:49.985533 13965 raft_consensus.cc:2267] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:49.985772 13965 ts_tablet_manager.cc:1905] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:49.987123 13965 ts_tablet_manager.cc:1918] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:49.987195 13965 log.cc:1198] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/wals/87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:49.987596 3649 catalog_manager.cc:4872] TS 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): tablet 87835ddc19954987b0f17f6aa0370c10 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:49.990914 13959 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:49.991194 13959 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap complete.
I20250112 21:06:49.991295 13959 ts_tablet_manager.cc:1397] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20250112 21:06:49.991408 13959 raft_consensus.cc:357] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:49.991458 13959 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:49.991500 13959 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:49.991648 13959 ts_tablet_manager.cc:1428] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:49.991834 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 1d1923072e1c45778fa5afdcfda8144d-102b9dd051ff4a2a8da8ff4692e56256 received from {username='slave'} at 127.26.82.68:58711
I20250112 21:06:49.991880 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 1d1923072e1c45778fa5afdcfda8144d-102b9dd051ff4a2a8da8ff4692e56256 on tablet 102b9dd051ff4a2a8da8ff4692e56256 with peer 1d1923072e1c45778fa5afdcfda8144d
I20250112 21:06:50.008509 3807 consensus_queue.cc:237] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:50.009421 5732 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.009449 5470 raft_consensus.cc:1270] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.009604 12262 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.009740 12262 consensus_queue.cc:1035] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.010315 14018 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.010421 5732 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.012490 3658 catalog_manager.cc:5526] T 917b9e7ed0ed43efa0a60ff461e4f213 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.013931 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 917b9e7ed0ed43efa0a60ff461e4f213 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:50.014041 13721 tablet_replica.cc:331] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:50.014103 13721 raft_consensus.cc:2238] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.014183 13721 raft_consensus.cc:2267] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.014968 13721 ts_tablet_manager.cc:1905] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.015501 5470 raft_consensus.cc:2949] T 917b9e7ed0ed43efa0a60ff461e4f213 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.016784 13721 ts_tablet_manager.cc:1918] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:50.016835 13721 log.cc:1198] T 917b9e7ed0ed43efa0a60ff461e4f213 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/917b9e7ed0ed43efa0a60ff461e4f213
I20250112 21:06:50.017218 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 917b9e7ed0ed43efa0a60ff461e4f213 (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:50.029016 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.038268 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.039613 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.051096 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.053653 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.054772 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.057405 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.061174 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.062587 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.065001 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.070343 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.071267 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.071841 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.078712 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.079983 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.080001 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.081424 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.091243 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.091425 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.093904 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.094192 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.100903 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.105823 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.123863 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.140225 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.140453 3936 raft_consensus.cc:1212] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20250112 21:06:50.146183 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.149305 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.149919 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.151244 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.151867 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.157485 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.161906 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.174612 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.174803 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.181514 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.182590 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.183343 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.184123 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.188382 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.192703 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.205736 5602 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:50.206194 5732 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:50.206383 12243 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:50.206415 4199 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:50.206825 12243 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:50.207132 12243 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:50.208246 5732 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:50.208369 3663 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:50.208438 4199 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:50.210107 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b6276ae83948b6b4ea2771d9ce8355 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:50.210281 14255 tablet_replica.cc:331] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:50.210392 14255 raft_consensus.cc:2238] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.210564 14255 raft_consensus.cc:2267] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.210915 14255 ts_tablet_manager.cc:1905] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.212637 14255 ts_tablet_manager.cc:1918] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:50.212770 14255 log.cc:1198] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/b3b6276ae83948b6b4ea2771d9ce8355
I20250112 21:06:50.213182 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet b3b6276ae83948b6b4ea2771d9ce8355 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:50.222415 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.230091 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.230542 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.243114 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.249882 4199 consensus_queue.cc:237] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:50.250492 5470 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.250591 5864 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.250814 6680 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.251077 6680 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.251785 13694 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:50.251914 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.252960 5470 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:50.253571 3656 catalog_manager.cc:5526] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:50.254292 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ba48c8dc7e4ee485f85ee94aca256b with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:50.254395 13721 tablet_replica.cc:331] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:50.254457 13721 raft_consensus.cc:2238] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.254516 13721 raft_consensus.cc:2267] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.254695 13721 ts_tablet_manager.cc:1905] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.255594 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.257087 13721 ts_tablet_manager.cc:1918] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:50.257134 13721 log.cc:1198] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/b6ba48c8dc7e4ee485f85ee94aca256b
I20250112 21:06:50.257490 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet b6ba48c8dc7e4ee485f85ee94aca256b (table rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]) successfully deleted
I20250112 21:06:50.258764 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.258951 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.263870 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.264173 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.269785 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.270483 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.276530 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.277143 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.279052 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.281266 5864 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:50.284029 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.308082 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.319754 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.320449 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.321187 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.322516 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.325508 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.329233 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.330472 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.332338 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.337015 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.341920 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.343185 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.343429 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.350268 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.358664 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.360487 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.390976 3936 consensus_queue.cc:237] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:50.392035 5470 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.392252 8535 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.392498 5732 raft_consensus.cc:1270] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.393023 8535 consensus_queue.cc:1035] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.394569 4199 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:50.394837 13972 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:50.395258 4067 raft_consensus.cc:1212] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20250112 21:06:50.398365 4067 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:50.398551 12231 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.398921 4067 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.399168 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.401566 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.401829 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.401901 8535 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.402046 5470 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.403131 3659 catalog_manager.cc:5526] T 90f8eabb6a0d4292953e3dec2efec97d P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.403800 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.404419 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 90f8eabb6a0d4292953e3dec2efec97d with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:50.404531 13721 tablet_replica.cc:331] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:50.404584 13721 raft_consensus.cc:2238] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.404639 13721 raft_consensus.cc:2267] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.404861 13721 ts_tablet_manager.cc:1905] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.406463 13721 ts_tablet_manager.cc:1918] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:50.406514 13721 log.cc:1198] T 90f8eabb6a0d4292953e3dec2efec97d P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/90f8eabb6a0d4292953e3dec2efec97d
I20250112 21:06:50.406896 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 90f8eabb6a0d4292953e3dec2efec97d (table rebalance_test_table_7 [id=3f42c0c2f1d146119b916da9e48b78d5]) successfully deleted
I20250112 21:06:50.410262 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.415846 5732 raft_consensus.cc:2949] T 90f8eabb6a0d4292953e3dec2efec97d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.419909 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.421356 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.422791 4067 raft_consensus.cc:1212] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.8->[2.9-2.9] Dedup: 2.9->[]
I20250112 21:06:50.423795 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.424764 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.429812 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.430296 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.434921 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.437017 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.441632 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.448371 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.449172 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.473989 3936 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:50.474627 5732 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:50.474615 4068 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:50.474825 8535 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:50.474928 14534 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:50.475636 8535 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.476791 5732 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.476897 3659 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 8 to 9, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.478907 4068 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.486017 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 with delete_type TABLET_DATA_TOMBSTONED (TS 38c8e8afd3f5411490f56ebf57aee109 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41748
I20250112 21:06:50.486224 14649 tablet_replica.cc:331] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: stopping tablet replica
I20250112 21:06:50.486366 14649 raft_consensus.cc:2238] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.486476 14649 raft_consensus.cc:2267] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.487951 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.490687 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.492825 14649 ts_tablet_manager.cc:1905] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.493844 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.497598 14649 ts_tablet_manager.cc:1918] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20250112 21:06:50.497663 14649 log.cc:1198] T 0d2ae40fed4b41218aa58c523d7d9785 P 38c8e8afd3f5411490f56ebf57aee109: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/wals/0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:50.498003 3650 catalog_manager.cc:4872] TS 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): tablet 0d2ae40fed4b41218aa58c523d7d9785 (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:50.501022 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.504971 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.509485 3936 consensus_queue.cc:237] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:50.509662 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.510010 4068 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:50.510017 5732 raft_consensus.cc:1270] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:50.510193 14520 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:50.510324 14534 consensus_queue.cc:1035] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:50.510427 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.511225 14520 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.511408 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.512452 4068 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.513557 3656 catalog_manager.cc:5526] T c93763a09f7d4460b1874bf328fb06f5 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.517400 5451 tablet_service.cc:1514] Processing DeleteTablet for tablet c93763a09f7d4460b1874bf328fb06f5 with delete_type TABLET_DATA_TOMBSTONED (TS 6f7f47d877a8407185013dba2959fddd not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51422
I20250112 21:06:50.517652 14668 tablet_replica.cc:331] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: stopping tablet replica
I20250112 21:06:50.517791 14668 raft_consensus.cc:2238] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.517905 14668 raft_consensus.cc:2267] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.518159 14668 ts_tablet_manager.cc:1905] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.519690 14668 ts_tablet_manager.cc:1918] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20250112 21:06:50.519835 14668 log.cc:1198] T c93763a09f7d4460b1874bf328fb06f5 P 6f7f47d877a8407185013dba2959fddd: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/wals/c93763a09f7d4460b1874bf328fb06f5
I20250112 21:06:50.520247 3651 catalog_manager.cc:4872] TS 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): tablet c93763a09f7d4460b1874bf328fb06f5 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:50.520787 5732 raft_consensus.cc:2949] T c93763a09f7d4460b1874bf328fb06f5 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:50.543496 4199 consensus_queue.cc:237] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:50.543962 5602 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.547848 3936 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.548383 13694 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.549053 13694 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.554576 13694 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:50.556223 4199 raft_consensus.cc:1212] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20250112 21:06:50.556730 3936 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:50.557943 5602 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:50.558151 3659 catalog_manager.cc:5526] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:50.558917 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet dd01159cf53642739a8a1dbcacbc933d with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:50.559027 13721 tablet_replica.cc:331] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:50.559082 13721 raft_consensus.cc:2238] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.559149 13721 raft_consensus.cc:2267] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.559310 13721 ts_tablet_manager.cc:1905] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.560984 13721 ts_tablet_manager.cc:1918] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:50.561046 13721 log.cc:1198] T dd01159cf53642739a8a1dbcacbc933d P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/dd01159cf53642739a8a1dbcacbc933d
I20250112 21:06:50.561385 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet dd01159cf53642739a8a1dbcacbc933d (table rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]) successfully deleted
I20250112 21:06:50.566494 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.571844 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.572775 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.573632 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.575351 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.582672 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.582975 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.584441 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.587298 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.588915 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.591620 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.596269 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.609055 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.610680 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.611572 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.614709 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.620334 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.621692 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.625288 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.626616 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.629532 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.635674 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.638927 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.644594 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.644642 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.648785 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.656416 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.658659 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.659687 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.660969 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.665037 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.671567 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.678033 4068 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256"
dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60688
I20250112 21:06:50.678134 4068 raft_consensus.cc:604] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:50.696291 4068 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:50.697002 5864 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20250112 21:06:50.697016 4002 consensus_peers.cc:487] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): Couldn't send request to peer 1d1923072e1c45778fa5afdcfda8144d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:50.697248 5732 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:50.697260 12524 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:50.697494 12524 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:50.698343 13315 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:50.699762 5732 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:50.699985 5864 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:50.700160 3657 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 7 to 8, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:50.703115 3936 consensus_queue.cc:237] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:50.703562 5732 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.703599 5470 raft_consensus.cc:1270] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:50.703827 14534 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.703979 14520 consensus_queue.cc:1035] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:50.704739 14520 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.706102 3657 catalog_manager.cc:5526] T 3fae2c2254d04e7a819f5d8638fd97ea P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.706367 5732 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.706897 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fae2c2254d04e7a819f5d8638fd97ea with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:50.708027 5470 raft_consensus.cc:2949] T 3fae2c2254d04e7a819f5d8638fd97ea P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.714785 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.718945 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.719543 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.720274 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.721136 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.721307 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.726637 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.737799 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.738224 14932 tablet_replica.cc:331] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:50.738293 14932 raft_consensus.cc:2238] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.739980 14932 raft_consensus.cc:2267] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.740178 14932 ts_tablet_manager.cc:1905] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.741955 14932 ts_tablet_manager.cc:1918] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:50.742022 14932 log.cc:1198] T 3fae2c2254d04e7a819f5d8638fd97ea P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/3fae2c2254d04e7a819f5d8638fd97ea
I20250112 21:06:50.742331 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 3fae2c2254d04e7a819f5d8638fd97ea (table rebalance_test_table_2 [id=5efd715f33104220b56c71d4cf8f47e6]) successfully deleted
I20250112 21:06:50.746804 5732 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:50.747370 4199 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:50.747422 5470 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
W20250112 21:06:50.747493 5667 consensus_peers.cc:487] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b -> Peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): Couldn't send request to peer bd4e9b42a6de4ff7a2d954e2be23c606. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:50.747594 12231 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:50.748008 12231 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:50.749948 12231 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:50.749945 5470 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:50.751634 4199 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:50.751686 3664 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:50.763540 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.764036 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.765120 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.768698 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.769078 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.774878 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.777247 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.781455 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.792738 3936 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:50.793857 5732 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20250112 21:06:50.793978 4068 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
W20250112 21:06:50.794273 3873 consensus_peers.cc:487] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d2ae40fed4b41218aa58c523d7d9785. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:50.794309 8535 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:50.794687 14520 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:50.795531 14520 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:50.796128 4068 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:50.797010 5732 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:50.797426 3658 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 9 to 10, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:50.801209 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.807209 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.808236 3787 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.812041 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.812850 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.813457 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.813623 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.816565 5732 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:50.817765 4199 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:50.818082 13972 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:50.821255 13972 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 7 to 9, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.823974 4199 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 7 to 9, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.824210 3662 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 7 to 9, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.826265 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 87835ddc19954987b0f17f6aa0370c10 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:50.826364 13721 tablet_replica.cc:331] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:50.826421 13721 raft_consensus.cc:2238] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:50.826516 13721 raft_consensus.cc:2267] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:50.826885 13721 ts_tablet_manager.cc:1905] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:50.827697 5732 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
W20250112 21:06:50.828205 5667 consensus_peers.cc:487] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b -> Peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): Couldn't send request to peer bd4e9b42a6de4ff7a2d954e2be23c606. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87835ddc19954987b0f17f6aa0370c10. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:50.828287 13721 ts_tablet_manager.cc:1918] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.8
I20250112 21:06:50.828347 13721 log.cc:1198] T 87835ddc19954987b0f17f6aa0370c10 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:50.828723 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 87835ddc19954987b0f17f6aa0370c10 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:50.828860 4199 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:50.828982 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.829097 14987 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:50.829671 14987 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:50.830583 3651 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 87835ddc19954987b0f17f6aa0370c10 with cas_config_opid_index 9: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:50.830735 4199 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:50.830739 3658 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 9 to 10, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:50.844597 3807 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0"
dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38360
I20250112 21:06:50.844691 3807 raft_consensus.cc:604] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:50.866341 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.870484 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.873699 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.878253 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.879130 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.879637 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.882844 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.887405 14018 raft_consensus.cc:988] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: : Instructing follower 2565f97e9cc54700ab2ce39aa7a546f8 to start an election
I20250112 21:06:50.887460 14018 raft_consensus.cc:1076] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Signalling peer 2565f97e9cc54700ab2ce39aa7a546f8 to start an election
I20250112 21:06:50.887610 4068 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0"
dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
from {username='slave'} at 127.26.82.65:50989
I20250112 21:06:50.887670 4068 raft_consensus.cc:491] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:50.887696 4068 raft_consensus.cc:3054] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:50.888442 4068 raft_consensus.cc:513] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:50.888599 4068 leader_election.cc:290] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655), 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:50.888736 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:50.888744 5732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
I20250112 21:06:50.888809 3807 raft_consensus.cc:3049] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:50.888810 5732 raft_consensus.cc:3054] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:50.888788 5602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ca3308b60c4367ab34f25ed5ecb0c0" candidate_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
I20250112 21:06:50.888834 3807 raft_consensus.cc:738] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:50.888855 5602 raft_consensus.cc:3054] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:50.888885 3807 consensus_queue.cc:260] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:50.888970 3807 raft_consensus.cc:3054] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:50.889557 5602 raft_consensus.cc:2463] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:50.889612 5732 raft_consensus.cc:2463] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:50.889709 3807 raft_consensus.cc:2463] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2565f97e9cc54700ab2ce39aa7a546f8 in term 2.
I20250112 21:06:50.889959 4004 leader_election.cc:304] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2565f97e9cc54700ab2ce39aa7a546f8, 67a5a7038ced40ddb6f45fdbfd99825c, 85a6e2a0c3b94dea8a094d396cfcac4b; no voters:
I20250112 21:06:50.890179 14923 raft_consensus.cc:2798] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:50.890242 14923 raft_consensus.cc:695] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Becoming Leader. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:50.890288 14923 consensus_queue.cc:237] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:50.890929 3663 catalog_manager.cc:5526] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: term changed from 1 to 2, leader changed from bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) to 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67). New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:50.905499 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.913089 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.916708 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.921550 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.922377 4049 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:50.923202 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.926872 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.927533 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.927654 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.931494 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:50.935307 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.941397 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.942142 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:50.943923 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.957468 4049 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:50.959362 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:50.961225 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:50.973150 15299 ts_tablet_manager.cc:927] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Initiating tablet copy from peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:50.973296 15299 tablet_copy_client.cc:323] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.70:43655
I20250112 21:06:50.973556 5622 tablet_copy_service.cc:140] P 67a5a7038ced40ddb6f45fdbfd99825c: Received BeginTabletCopySession request for tablet 2e6bf2e45872401f8245a586b6fe7431 from peer fd8fa04957924e9f83afd61f9adfb455 ({username='slave'} at 127.26.82.66:37129)
I20250112 21:06:50.973639 5622 tablet_copy_service.cc:161] P 67a5a7038ced40ddb6f45fdbfd99825c: Beginning new tablet copy session on tablet 2e6bf2e45872401f8245a586b6fe7431 from peer fd8fa04957924e9f83afd61f9adfb455 at {username='slave'} at 127.26.82.66:37129: session id = fd8fa04957924e9f83afd61f9adfb455-2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:50.974179 5622 tablet_copy_source_session.cc:215] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:50.974519 15299 ts_tablet_manager.cc:1905] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:50.976853 15299 ts_tablet_manager.cc:1918] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 2.8
I20250112 21:06:50.976954 15299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6bf2e45872401f8245a586b6fe7431. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:50.977689 15299 tablet_copy_client.cc:806] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:50.977790 15299 tablet_copy_client.cc:670] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:50.979231 15299 tablet_copy_client.cc:538] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:50.980541 15299 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:50.999162 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.004155 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.008136 15299 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:51.008354 15299 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap complete.
I20250112 21:06:51.008468 15299 ts_tablet_manager.cc:1397] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.004s
I20250112 21:06:51.008567 15299 raft_consensus.cc:357] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.008626 15299 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:51.009056 15299 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.009224 15299 ts_tablet_manager.cc:1428] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250112 21:06:51.009292 4049 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20250112 21:06:51.009495 5622 tablet_copy_service.cc:342] P 67a5a7038ced40ddb6f45fdbfd99825c: Request end of tablet copy session fd8fa04957924e9f83afd61f9adfb455-2e6bf2e45872401f8245a586b6fe7431 received from {username='slave'} at 127.26.82.66:37129
I20250112 21:06:51.009548 5622 tablet_copy_service.cc:434] P 67a5a7038ced40ddb6f45fdbfd99825c: ending tablet copy session fd8fa04957924e9f83afd61f9adfb455-2e6bf2e45872401f8245a586b6fe7431 on tablet 2e6bf2e45872401f8245a586b6fe7431 with peer fd8fa04957924e9f83afd61f9adfb455
I20250112 21:06:51.012143 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.012811 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.022104 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.026453 14923 raft_consensus.cc:988] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: : Instructing follower 1d1923072e1c45778fa5afdcfda8144d to start an election
I20250112 21:06:51.026522 14923 raft_consensus.cc:1076] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Signalling peer 1d1923072e1c45778fa5afdcfda8144d to start an election
I20250112 21:06:51.026806 4199 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256"
dest_uuid: "1d1923072e1c45778fa5afdcfda8144d"
from {username='slave'} at 127.26.82.67:58023
I20250112 21:06:51.026877 4199 raft_consensus.cc:491] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:51.026902 4199 raft_consensus.cc:3054] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.027613 4199 raft_consensus.cc:513] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:51.027741 4199 leader_election.cc:290] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 2 election: Requested vote from peers 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119), 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429), 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:51.027935 4068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "2565f97e9cc54700ab2ce39aa7a546f8"
I20250112 21:06:51.028029 4068 raft_consensus.cc:3049] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:51.028055 4068 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Running, Role: LEADER
I20250112 21:06:51.028106 4068 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:51.028185 4068 raft_consensus.cc:3054] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.028543 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
I20250112 21:06:51.028610 5864 raft_consensus.cc:3054] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.028796 5602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b9dd051ff4a2a8da8ff4692e56256" candidate_uuid: "1d1923072e1c45778fa5afdcfda8144d" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
I20250112 21:06:51.028857 5602 raft_consensus.cc:3054] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.029137 4178 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.029297 5864 raft_consensus.cc:2463] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 2.
I20250112 21:06:51.029515 5602 raft_consensus.cc:2463] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 2.
I20250112 21:06:51.029544 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.030721 4133 leader_election.cc:304] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1d1923072e1c45778fa5afdcfda8144d, 38c8e8afd3f5411490f56ebf57aee109, 67a5a7038ced40ddb6f45fdbfd99825c; no voters:
I20250112 21:06:51.030776 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.030817 14771 raft_consensus.cc:2798] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:51.030854 14771 raft_consensus.cc:695] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 2 LEADER]: Becoming Leader. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Running, Role: LEADER
I20250112 21:06:51.030936 14771 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:51.031603 3658 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: term changed from 1 to 2, leader changed from 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) to 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68). New cstate: current_term: 2 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:51.032706 4068 raft_consensus.cc:2463] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d1923072e1c45778fa5afdcfda8144d in term 2.
I20250112 21:06:51.034330 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.035215 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.036221 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.036453 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.038733 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.044246 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.048972 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.060817 5470 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:51.061419 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:51.061600 13254 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:51.062287 4068 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:51.062471 13254 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:51.062896 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.064062 13254 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.064563 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.065210 4068 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.066409 3658 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.069650 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e with delete_type TABLET_DATA_TOMBSTONED (TS 38c8e8afd3f5411490f56ebf57aee109 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:41748
I20250112 21:06:51.071368 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.076771 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.082186 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.082731 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.091274 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.091610 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.095340 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.096200 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.096292 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.099867 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.101172 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.106240 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.108295 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.114280 15410 tablet_replica.cc:331] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: stopping tablet replica
I20250112 21:06:51.114360 15410 raft_consensus.cc:2238] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:51.114462 15410 raft_consensus.cc:2267] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:51.114710 15410 ts_tablet_manager.cc:1905] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:51.115954 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.116370 15410 ts_tablet_manager.cc:1918] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20250112 21:06:51.116428 15410 log.cc:1198] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/wals/de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:51.116746 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.117491 3650 catalog_manager.cc:4872] TS 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): tablet de6b6ac6eeff431b9ac8cce47ea3a59e (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:51.127718 4068 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.128230 5864 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:51.128247 5732 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
W20250112 21:06:51.128357 4003 consensus_peers.cc:487] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): Couldn't send request to peer fd8fa04957924e9f83afd61f9adfb455. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:51.128439 14923 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:51.128590 13312 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:51.129264 13312 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.129403 5864 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.130577 3659 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 8 to 9, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.130829 5732 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.135372 15469 ts_tablet_manager.cc:927] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Initiating tablet copy from peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:51.135550 15469 tablet_copy_client.cc:323] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.71:37889
I20250112 21:06:51.135861 5753 tablet_copy_service.cc:140] P 85a6e2a0c3b94dea8a094d396cfcac4b: Received BeginTabletCopySession request for tablet 345f4bf45767475ca481525f3d26a34a from peer bd4e9b42a6de4ff7a2d954e2be23c606 ({username='slave'} at 127.26.82.65:37143)
I20250112 21:06:51.135937 5753 tablet_copy_service.cc:161] P 85a6e2a0c3b94dea8a094d396cfcac4b: Beginning new tablet copy session on tablet 345f4bf45767475ca481525f3d26a34a from peer bd4e9b42a6de4ff7a2d954e2be23c606 at {username='slave'} at 127.26.82.65:37143: session id = bd4e9b42a6de4ff7a2d954e2be23c606-345f4bf45767475ca481525f3d26a34a
I20250112 21:06:51.136420 5753 tablet_copy_source_session.cc:215] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:51.136883 15469 ts_tablet_manager.cc:1905] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:51.138409 15469 ts_tablet_manager.cc:1918] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 2.7
I20250112 21:06:51.138509 15469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 345f4bf45767475ca481525f3d26a34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:51.139196 15469 tablet_copy_client.cc:806] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:51.139314 15469 tablet_copy_client.cc:670] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:51.140648 15469 tablet_copy_client.cc:538] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:51.141395 15469 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:51.148898 15469 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:51.149112 15469 tablet_bootstrap.cc:492] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap complete.
I20250112 21:06:51.149194 15469 ts_tablet_manager.cc:1397] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20250112 21:06:51.149286 15469 raft_consensus.cc:357] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:51.149348 15469 raft_consensus.cc:738] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:51.149397 15469 consensus_queue.cc:260] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:51.149682 15469 ts_tablet_manager.cc:1428] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:51.150921 5753 tablet_copy_service.cc:342] P 85a6e2a0c3b94dea8a094d396cfcac4b: Request end of tablet copy session bd4e9b42a6de4ff7a2d954e2be23c606-345f4bf45767475ca481525f3d26a34a received from {username='slave'} at 127.26.82.65:37143
I20250112 21:06:51.150980 5753 tablet_copy_service.cc:434] P 85a6e2a0c3b94dea8a094d396cfcac4b: ending tablet copy session bd4e9b42a6de4ff7a2d954e2be23c606-345f4bf45767475ca481525f3d26a34a on tablet 345f4bf45767475ca481525f3d26a34a with peer bd4e9b42a6de4ff7a2d954e2be23c606
I20250112 21:06:51.157011 4199 consensus_queue.cc:237] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:51.157651 5602 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:51.157851 5470 raft_consensus.cc:1270] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20250112 21:06:51.158079 14771 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:51.158210 14771 consensus_queue.cc:1035] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:51.158236 15525 ts_tablet_manager.cc:927] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:51.158470 15525 tablet_copy_client.cc:323] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:51.158680 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 7249078cad3e4bc3ab2b62d8887987df from peer 1d1923072e1c45778fa5afdcfda8144d ({username='slave'} at 127.26.82.68:58711)
I20250112 21:06:51.158727 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 7249078cad3e4bc3ab2b62d8887987df from peer 1d1923072e1c45778fa5afdcfda8144d at {username='slave'} at 127.26.82.68:58711: session id = 1d1923072e1c45778fa5afdcfda8144d-7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:51.159302 4089 tablet_copy_source_session.cc:215] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:51.159293 6680 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:51.159859 15525 ts_tablet_manager.cc:1905] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:51.160326 5470 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:51.161059 3657 catalog_manager.cc:5526] T 979cf479af044430904e610adee50359 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:51.161937 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 979cf479af044430904e610adee50359 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:51.162071 13721 tablet_replica.cc:331] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:51.162153 13721 raft_consensus.cc:2238] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:51.162225 13721 raft_consensus.cc:2267] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:51.162415 13721 ts_tablet_manager.cc:1905] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:51.164005 13721 ts_tablet_manager.cc:1918] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20250112 21:06:51.164053 13721 log.cc:1198] T 979cf479af044430904e610adee50359 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/979cf479af044430904e610adee50359
I20250112 21:06:51.164402 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 979cf479af044430904e610adee50359 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
I20250112 21:06:51.166178 15525 ts_tablet_manager.cc:1918] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20250112 21:06:51.166333 15525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7249078cad3e4bc3ab2b62d8887987df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:51.167258 15525 tablet_copy_client.cc:806] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:51.167415 15525 tablet_copy_client.cc:670] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:51.167587 5602 raft_consensus.cc:2949] T 979cf479af044430904e610adee50359 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:51.170385 15525 tablet_copy_client.cc:538] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:51.171064 15525 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:51.176597 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.177862 15525 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:51.178117 15525 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap complete.
I20250112 21:06:51.178201 15525 ts_tablet_manager.cc:1397] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20250112 21:06:51.178297 15525 raft_consensus.cc:357] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:51.178367 15525 raft_consensus.cc:738] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:51.178417 15525 consensus_queue.cc:260] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:51.178530 15525 ts_tablet_manager.cc:1428] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:51.180840 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.181375 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session 1d1923072e1c45778fa5afdcfda8144d-7249078cad3e4bc3ab2b62d8887987df received from {username='slave'} at 127.26.82.68:58711
I20250112 21:06:51.181429 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session 1d1923072e1c45778fa5afdcfda8144d-7249078cad3e4bc3ab2b62d8887987df on tablet 7249078cad3e4bc3ab2b62d8887987df with peer 1d1923072e1c45778fa5afdcfda8144d
I20250112 21:06:51.185971 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.188493 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.188783 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.188815 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.194216 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.200537 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.201746 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.205433 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.212600 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.216286 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.217046 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.218897 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.225252 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.232462 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.232775 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.234392 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.241051 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.242384 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.242522 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.245838 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.255692 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.258231 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.296486 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.303584 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.310101 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.313292 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.313668 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.317839 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.321017 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.321281 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.324615 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.327734 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.328058 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.329116 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.332808 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.334314 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.335103 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.346032 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.361105 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.363955 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.368266 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.369455 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.371295 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.373355 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.377344 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.383436 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.386241 15755 ts_tablet_manager.cc:927] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:51.386409 15755 tablet_copy_client.cc:323] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:51.387913 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:53179)
I20250112 21:06:51.387986 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:53179: session id = 6f7f47d877a8407185013dba2959fddd-0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:51.388513 3958 tablet_copy_source_session.cc:215] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:51.388979 15755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:51.390389 15755 tablet_copy_client.cc:806] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:51.390556 15755 tablet_copy_client.cc:670] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:51.392071 15755 tablet_copy_client.cc:538] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:51.392835 15755 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:51.396713 15755 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:51.396939 15755 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:51.397025 15755 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20250112 21:06:51.397126 15755 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:51.397181 15755 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:51.397224 15755 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:51.397425 15755 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:51.397697 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-0d2ae40fed4b41218aa58c523d7d9785 received from {username='slave'} at 127.26.82.69:53179
I20250112 21:06:51.397753 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-0d2ae40fed4b41218aa58c523d7d9785 on tablet 0d2ae40fed4b41218aa58c523d7d9785 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:51.404948 15469 ts_tablet_manager.cc:927] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: Initiating tablet copy from peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:51.405129 15469 tablet_copy_client.cc:323] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.71:37889
I20250112 21:06:51.405689 5753 tablet_copy_service.cc:140] P 85a6e2a0c3b94dea8a094d396cfcac4b: Received BeginTabletCopySession request for tablet 87835ddc19954987b0f17f6aa0370c10 from peer bd4e9b42a6de4ff7a2d954e2be23c606 ({username='slave'} at 127.26.82.65:37143)
I20250112 21:06:51.405771 5753 tablet_copy_service.cc:161] P 85a6e2a0c3b94dea8a094d396cfcac4b: Beginning new tablet copy session on tablet 87835ddc19954987b0f17f6aa0370c10 from peer bd4e9b42a6de4ff7a2d954e2be23c606 at {username='slave'} at 127.26.82.65:37143: session id = bd4e9b42a6de4ff7a2d954e2be23c606-87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:51.406459 5753 tablet_copy_source_session.cc:215] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:51.406894 15469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:51.407791 15469 tablet_copy_client.cc:806] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:51.407871 15469 tablet_copy_client.cc:670] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:51.408785 15469 tablet_copy_client.cc:538] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:51.409580 15469 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:51.413663 5602 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:51.413883 15469 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:51.413889 14923 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:51.414180 15469 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap complete.
I20250112 21:06:51.414341 15469 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20250112 21:06:51.414633 15469 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:51.414779 15469 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:51.415014 15469 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:51.415530 15469 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:51.416661 5753 tablet_copy_service.cc:342] P 85a6e2a0c3b94dea8a094d396cfcac4b: Request end of tablet copy session bd4e9b42a6de4ff7a2d954e2be23c606-87835ddc19954987b0f17f6aa0370c10 received from {username='slave'} at 127.26.82.65:37143
I20250112 21:06:51.416752 5753 tablet_copy_service.cc:434] P 85a6e2a0c3b94dea8a094d396cfcac4b: ending tablet copy session bd4e9b42a6de4ff7a2d954e2be23c606-87835ddc19954987b0f17f6aa0370c10 on tablet 87835ddc19954987b0f17f6aa0370c10 with peer bd4e9b42a6de4ff7a2d954e2be23c606
I20250112 21:06:51.420049 3807 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:51.420439 14923 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:51.421072 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.421272 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.421864 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.422262 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.431790 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.433261 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.438586 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.442461 5732 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:51.443105 12524 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:51.443658 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.452582 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.453843 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.454139 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.457449 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.457789 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.459151 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.468936 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.468984 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.490060 4199 consensus_queue.cc:237] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.490612 3936 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:51.490912 6680 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:51.491640 14771 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.491777 3936 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.493041 3663 catalog_manager.cc:5526] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 7 to 8, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.493970 3936 raft_consensus.cc:1212] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.9->[2.10-2.10] Dedup: 2.10->[]
I20250112 21:06:51.494771 5582 tablet_service.cc:1514] Processing DeleteTablet for tablet dd01159cf53642739a8a1dbcacbc933d with delete_type TABLET_DATA_TOMBSTONED (TS 67a5a7038ced40ddb6f45fdbfd99825c not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:49494
I20250112 21:06:51.495038 4199 consensus_queue.cc:237] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
W20250112 21:06:51.495553 4135 consensus_peers.cc:487] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd01159cf53642739a8a1dbcacbc933d. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:51.495594 15884 tablet_replica.cc:331] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: stopping tablet replica
I20250112 21:06:51.495659 15884 raft_consensus.cc:2238] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:51.495736 15884 raft_consensus.cc:2267] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:51.495740 3936 raft_consensus.cc:1270] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:51.495896 14771 consensus_queue.cc:1035] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:51.496001 15884 ts_tablet_manager.cc:1905] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:51.496474 14771 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:51.497521 3649 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet dd01159cf53642739a8a1dbcacbc933d with cas_config_opid_index 8: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:51.497586 3658 catalog_manager.cc:5526] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 8 to 9, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:51.497665 3936 raft_consensus.cc:2949] T dd01159cf53642739a8a1dbcacbc933d P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:51.497790 15884 ts_tablet_manager.cc:1918] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.7
I20250112 21:06:51.497843 15884 log.cc:1198] T dd01159cf53642739a8a1dbcacbc933d P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/wals/dd01159cf53642739a8a1dbcacbc933d
I20250112 21:06:51.498194 3649 catalog_manager.cc:4872] TS 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): tablet dd01159cf53642739a8a1dbcacbc933d (table rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]) successfully deleted
I20250112 21:06:51.513410 3807 consensus_queue.cc:237] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:51.514096 5864 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:51.514411 5602 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:51.514809 15207 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20250112 21:06:51.514902 3740 consensus_peers.cc:487] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 -> Peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): Couldn't send request to peer 2565f97e9cc54700ab2ce39aa7a546f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd9176ff75364d81a5b3bdd0e7e10673. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:51.515022 15207 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:51.515595 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.515631 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.518707 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.521205 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.526075 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.527033 3807 raft_consensus.cc:1212] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.8->[2.9-2.9] Dedup: 2.9->[]
I20250112 21:06:51.528347 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.530969 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:51.531235 15894 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:51.532018 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.532464 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.532473 15207 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.532842 5602 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.534201 3663 catalog_manager.cc:5526] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.536314 5863 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:51.538260 5602 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:51.538455 15894 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:51.542351 4068 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:51.542810 14771 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:51.562458 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.566776 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.568495 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.578481 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.578742 5470 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:51.579254 4068 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20250112 21:06:51.579437 15409 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:51.579501 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.580030 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
W20250112 21:06:51.583498 5405 consensus_peers.cc:487] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:51.583506 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20250112 21:06:51.583732 16014 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:51.584322 15409 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:51.585731 4068 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:51.585762 3658 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:51.589290 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:51.589639 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.590190 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.597141 16029 ts_tablet_manager.cc:927] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:51.597316 16029 tablet_copy_client.cc:323] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:51.597538 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 3c95203cf74b4b90867a9a3a334dbb21 from peer fd8fa04957924e9f83afd61f9adfb455 ({username='slave'} at 127.26.82.66:38619)
I20250112 21:06:51.597584 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 3c95203cf74b4b90867a9a3a334dbb21 from peer fd8fa04957924e9f83afd61f9adfb455 at {username='slave'} at 127.26.82.66:38619: session id = fd8fa04957924e9f83afd61f9adfb455-3c95203cf74b4b90867a9a3a334dbb21
I20250112 21:06:51.598105 4089 tablet_copy_source_session.cc:215] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:51.598479 16029 ts_tablet_manager.cc:1905] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:51.600484 16029 ts_tablet_manager.cc:1918] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 2.7
I20250112 21:06:51.600639 16029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c95203cf74b4b90867a9a3a334dbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:51.603484 16029 tablet_copy_client.cc:806] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:51.603652 16029 tablet_copy_client.cc:670] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:51.604735 16029 tablet_copy_client.cc:538] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:51.605391 16029 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:51.612473 16029 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:51.613091 16029 tablet_bootstrap.cc:492] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Bootstrap complete.
I20250112 21:06:51.613221 16029 ts_tablet_manager.cc:1397] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20250112 21:06:51.613341 16029 raft_consensus.cc:357] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.613432 16029 raft_consensus.cc:738] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:51.613502 16029 consensus_queue.cc:260] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.613878 16029 ts_tablet_manager.cc:1428] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:51.614121 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session fd8fa04957924e9f83afd61f9adfb455-3c95203cf74b4b90867a9a3a334dbb21 received from {username='slave'} at 127.26.82.66:38619
I20250112 21:06:51.614183 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session fd8fa04957924e9f83afd61f9adfb455-3c95203cf74b4b90867a9a3a334dbb21 on tablet 3c95203cf74b4b90867a9a3a334dbb21 with peer fd8fa04957924e9f83afd61f9adfb455
I20250112 21:06:51.620268 4199 raft_consensus.cc:1212] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20250112 21:06:51.634263 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.642359 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.643956 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.648738 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.648864 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.651208 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.652743 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.657778 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.658865 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.660612 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.663944 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.664474 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.671566 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.672542 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.680153 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.684393 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.685511 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.689253 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.690894 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.696085 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.702219 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.705485 3787 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:51.706784 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.710701 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.724932 4199 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:51.725562 5602 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:51.725616 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:51.725919 6680 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:51.726027 6680 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:51.726608 15894 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:51.726764 5864 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:51.728060 3807 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0"
dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38652
I20250112 21:06:51.728138 3807 raft_consensus.cc:604] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Received request to transfer leadership
I20250112 21:06:51.728085 3661 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 8 to 10, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 2 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:51.728132 5602 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:51.728885 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet 102b9dd051ff4a2a8da8ff4692e56256 with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:51.731170 16196 tablet_replica.cc:331] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:51.731245 16196 raft_consensus.cc:2238] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:51.731289 16196 raft_consensus.cc:2267] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:51.731526 16196 ts_tablet_manager.cc:1905] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:51.732362 5602 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:51.732847 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20250112 21:06:51.732997 16196 ts_tablet_manager.cc:1918] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20250112 21:06:51.733070 16196 log.cc:1198] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:51.733104 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:51.733376 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet 102b9dd051ff4a2a8da8ff4692e56256 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:51.734215 12553 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:51.735114 5470 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:51.736209 3665 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 10 to 11, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:51.738948 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:51.739184 5602 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.739522 3936 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:51.739719 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:51.740010 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:51.740260 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:51.740489 3936 raft_consensus.cc:867] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }.
Pending config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }:
New config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
W20250112 21:06:51.740537 3936 raft_consensus.cc:1574] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 NON_PARTICIPANT]: Could not prepare op 'term: 2 index: 12' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }.
Pending config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }:
New config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:51.740716 3936 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 NON_PARTICIPANT]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:51.741369 12553 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.741467 5470 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.742385 3649 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 2e6bf2e45872401f8245a586b6fe7431 with cas_config_opid_index 11: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:51.742478 3664 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 11 to 12, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:51.744896 3936 raft_consensus.cc:1476] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20250112 21:06:51.745023 3936 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
W20250112 21:06:51.746284 3650 catalog_manager.cc:4846] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): delete failed for tablet 2e6bf2e45872401f8245a586b6fe7431 because of a CAS failure. No further retry: Illegal state: Request specified cas_config_opid_index_less_or_equal of 10 but the committed config has opid_index of 12
I20250112 21:06:51.746551 15936 raft_consensus.cc:988] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: : Instructing follower 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:51.746620 15936 raft_consensus.cc:1076] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Signalling peer 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:51.746834 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.746832 5602 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0"
dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
from {username='slave'} at 127.26.82.65:54203
I20250112 21:06:51.746889 5602 raft_consensus.cc:491] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:51.746914 5602 raft_consensus.cc:3054] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.747543 5602 raft_consensus.cc:513] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:51.747694 5602 leader_election.cc:290] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 2 election: Requested vote from peers fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:51.748764 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:51.748852 3807 raft_consensus.cc:3049] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 21:06:51.748880 3807 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Running, Role: LEADER
I20250112 21:06:51.748943 3807 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:51.749055 3807 raft_consensus.cc:3054] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.749830 3807 raft_consensus.cc:2463] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:51.750689 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.751220 3936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "fd8fa04957924e9f83afd61f9adfb455"
I20250112 21:06:51.751310 3936 raft_consensus.cc:3054] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.751930 3936 raft_consensus.cc:2463] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:51.752295 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:51.752840 5536 leader_election.cc:304] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 67a5a7038ced40ddb6f45fdbfd99825c, bd4e9b42a6de4ff7a2d954e2be23c606, fd8fa04957924e9f83afd61f9adfb455; no voters:
I20250112 21:06:51.753055 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.754655 5582 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.755923 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.758195 16217 raft_consensus.cc:2798] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 21:06:51.758416 16217 raft_consensus.cc:695] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Becoming Leader. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Running, Role: LEADER
I20250112 21:06:51.758600 16217 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:51.759737 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8880a7467c6a44da96f98c3cc368c9f0" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
I20250112 21:06:51.759816 5864 raft_consensus.cc:3054] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Advancing to term 2
I20250112 21:06:51.760041 3663 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: term changed from 1 to 2, leader changed from bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) to 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70). New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:51.760434 5864 raft_consensus.cc:2463] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 2.
I20250112 21:06:51.760600 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.768949 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.798419 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.798764 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.801003 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.805590 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:51.807597 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.808542 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:51.809545 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.822292 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.830063 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.831354 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.836712 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.837299 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:51.844514 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.845028 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.845232 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.853519 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:51.871932 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.874840 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.882337 3807 raft_consensus.cc:1212] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.9->[2.10-2.10] Dedup: 2.10->[]
I20250112 21:06:51.882530 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:51.883538 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:51.883626 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.885075 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.885823 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.891137 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.910804 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.912619 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.914387 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:51.921412 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.922679 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:51.922993 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:51.925077 4068 consensus_queue.cc:237] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:51.925577 5732 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:51.926009 12524 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:51.926236 5602 raft_consensus.cc:1270] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:51.926569 12524 consensus_queue.cc:1035] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:51.927307 12524 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:51.927476 5732 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:51.928690 3664 catalog_manager.cc:5526] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:51.928740 5602 raft_consensus.cc:2949] T 47ca3308b60c4367ab34f25ed5ecb0c0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:51.929632 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ca3308b60c4367ab34f25ed5ecb0c0 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:51.933081 16415 tablet_replica.cc:331] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:51.933194 16415 raft_consensus.cc:2238] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:51.933295 16415 raft_consensus.cc:2267] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:51.933503 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:51.933845 16415 ts_tablet_manager.cc:1905] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:51.934013 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:51.935628 16415 ts_tablet_manager.cc:1918] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:51.935709 16415 log.cc:1198] T 47ca3308b60c4367ab34f25ed5ecb0c0 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/47ca3308b60c4367ab34f25ed5ecb0c0
I20250112 21:06:51.936064 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 47ca3308b60c4367ab34f25ed5ecb0c0 (table rebalance_test_table_8 [id=ef466625e9a14b348d82de0c47adfc6b]) successfully deleted
I20250112 21:06:51.958421 16479 ts_tablet_manager.cc:927] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:51.958560 16479 tablet_copy_client.cc:323] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.69:40871
I20250112 21:06:51.961603 5485 tablet_copy_service.cc:140] P 6f7f47d877a8407185013dba2959fddd: Received BeginTabletCopySession request for tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:40553)
I20250112 21:06:51.961664 5485 tablet_copy_service.cc:161] P 6f7f47d877a8407185013dba2959fddd: Beginning new tablet copy session on tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:40553: session id = 38c8e8afd3f5411490f56ebf57aee109-de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:51.962106 5485 tablet_copy_source_session.cc:215] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:51.962352 16479 ts_tablet_manager.cc:1905] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:51.964001 16479 ts_tablet_manager.cc:1918] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 2.9
I20250112 21:06:51.964098 16479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:51.964746 16479 tablet_copy_client.cc:806] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:51.964852 16479 tablet_copy_client.cc:670] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:51.965734 16479 tablet_copy_client.cc:538] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:51.966212 16479 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:51.974656 16479 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:51.974895 16479 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:51.975008 16479 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20250112 21:06:51.975131 16479 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:51.975208 16479 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:51.975276 16479 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:51.975519 16479 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.003s sys 0.000s
I20250112 21:06:51.975766 5485 tablet_copy_service.cc:342] P 6f7f47d877a8407185013dba2959fddd: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-de6b6ac6eeff431b9ac8cce47ea3a59e received from {username='slave'} at 127.26.82.72:40553
I20250112 21:06:51.975831 5485 tablet_copy_service.cc:434] P 6f7f47d877a8407185013dba2959fddd: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-de6b6ac6eeff431b9ac8cce47ea3a59e on tablet de6b6ac6eeff431b9ac8cce47ea3a59e with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:51.999439 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.001505 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.006726 3936 raft_consensus.cc:1212] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.8->[2.9-2.9] Dedup: 2.9->[]
I20250112 21:06:52.007058 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.010005 5470 raft_consensus.cc:1212] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.9->[1.10-1.10] Dedup: 1.10->[]
I20250112 21:06:52.011979 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.019882 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.022761 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.027525 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.027896 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.035796 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.036103 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.036492 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.041886 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.048241 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.048635 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.051434 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.053100 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.066025 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.067821 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.074672 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.076539 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.082152 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.085937 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.086910 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.091251 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.098522 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.101754 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.106551 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.108014 16639 ts_tablet_manager.cc:927] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: Initiating tablet copy from peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903)
I20250112 21:06:52.108139 16639 tablet_copy_client.cc:323] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.65:42903
I20250112 21:06:52.108384 3827 tablet_copy_service.cc:140] P bd4e9b42a6de4ff7a2d954e2be23c606: Received BeginTabletCopySession request for tablet dd9176ff75364d81a5b3bdd0e7e10673 from peer 2565f97e9cc54700ab2ce39aa7a546f8 ({username='slave'} at 127.26.82.67:41905)
I20250112 21:06:52.108436 3827 tablet_copy_service.cc:161] P bd4e9b42a6de4ff7a2d954e2be23c606: Beginning new tablet copy session on tablet dd9176ff75364d81a5b3bdd0e7e10673 from peer 2565f97e9cc54700ab2ce39aa7a546f8 at {username='slave'} at 127.26.82.67:41905: session id = 2565f97e9cc54700ab2ce39aa7a546f8-dd9176ff75364d81a5b3bdd0e7e10673
I20250112 21:06:52.109004 3827 tablet_copy_source_session.cc:215] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:52.109524 16639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd9176ff75364d81a5b3bdd0e7e10673. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:52.110916 16639 tablet_copy_client.cc:806] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:52.111032 16639 tablet_copy_client.cc:670] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:52.111718 16639 tablet_copy_client.cc:538] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:52.112452 16639 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:52.114269 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.116398 16639 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:52.116590 16639 tablet_bootstrap.cc:492] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap complete.
I20250112 21:06:52.116817 4199 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:52.117270 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
W20250112 21:06:52.117384 4133 consensus_peers.cc:487] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d -> Peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): Couldn't send request to peer 2565f97e9cc54700ab2ce39aa7a546f8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:52.117422 5601 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20250112 21:06:52.117612 15894 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:52.117713 14771 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:52.117853 16639 ts_tablet_manager.cc:1397] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250112 21:06:52.118316 16639 raft_consensus.cc:357] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:52.118399 16639 raft_consensus.cc:738] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:52.118515 16639 consensus_queue.cc:260] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:52.118773 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.119026 16639 ts_tablet_manager.cc:1428] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:52.119105 14771 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:52.119294 3827 tablet_copy_service.cc:342] P bd4e9b42a6de4ff7a2d954e2be23c606: Request end of tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-dd9176ff75364d81a5b3bdd0e7e10673 received from {username='slave'} at 127.26.82.67:41905
I20250112 21:06:52.119336 3827 tablet_copy_service.cc:434] P bd4e9b42a6de4ff7a2d954e2be23c606: ending tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-dd9176ff75364d81a5b3bdd0e7e10673 on tablet dd9176ff75364d81a5b3bdd0e7e10673 with peer 2565f97e9cc54700ab2ce39aa7a546f8
I20250112 21:06:52.119647 16648 ts_tablet_manager.cc:927] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:52.119766 16648 tablet_copy_client.cc:323] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:52.120417 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet dd01159cf53642739a8a1dbcacbc933d from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:49797)
I20250112 21:06:52.120466 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet dd01159cf53642739a8a1dbcacbc933d from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:49797: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-dd01159cf53642739a8a1dbcacbc933d
I20250112 21:06:52.120479 3664 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 10 to 11, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New cstate: current_term: 2 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:52.120776 5601 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:52.121038 4220 tablet_copy_source_session.cc:215] T dd01159cf53642739a8a1dbcacbc933d P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:52.121459 16648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd01159cf53642739a8a1dbcacbc933d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:52.122784 16648 tablet_copy_client.cc:806] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:52.122884 16648 tablet_copy_client.cc:670] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:52.123250 5864 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:52.124639 16648 tablet_copy_client.cc:538] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:52.125325 16648 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:52.128486 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.128854 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.133473 16648 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:52.133666 16648 tablet_bootstrap.cc:492] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:52.133780 16648 ts_tablet_manager.cc:1397] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20250112 21:06:52.133890 16648 raft_consensus.cc:357] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:52.133958 16648 raft_consensus.cc:738] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:52.134013 16648 consensus_queue.cc:260] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:52.134109 16648 ts_tablet_manager.cc:1428] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20250112 21:06:52.134462 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.135795 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-dd01159cf53642739a8a1dbcacbc933d received from {username='slave'} at 127.26.82.71:49797
I20250112 21:06:52.135851 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-dd01159cf53642739a8a1dbcacbc933d on tablet dd01159cf53642739a8a1dbcacbc933d with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:52.142515 3807 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:52.143424 16217 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:52.146054 4068 consensus_queue.cc:237] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:52.146569 3936 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:52.146817 13312 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:52.147107 5732 raft_consensus.cc:1270] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:52.147317 13312 consensus_queue.cc:1035] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:52.149644 13315 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.149892 5732 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.151046 3658 catalog_manager.cc:5526] T 3c95203cf74b4b90867a9a3a334dbb21 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New cstate: current_term: 2 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.152521 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c95203cf74b4b90867a9a3a334dbb21 with delete_type TABLET_DATA_TOMBSTONED (TS 38c8e8afd3f5411490f56ebf57aee109 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:41748
I20250112 21:06:52.152688 3936 raft_consensus.cc:2949] T 3c95203cf74b4b90867a9a3a334dbb21 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.153342 16710 tablet_replica.cc:331] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: stopping tablet replica
I20250112 21:06:52.153422 16710 raft_consensus.cc:2238] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.153542 16710 raft_consensus.cc:2267] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:52.153719 16710 ts_tablet_manager.cc:1905] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.155668 16710 ts_tablet_manager.cc:1918] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20250112 21:06:52.155874 16710 log.cc:1198] T 3c95203cf74b4b90867a9a3a334dbb21 P 38c8e8afd3f5411490f56ebf57aee109: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/wals/3c95203cf74b4b90867a9a3a334dbb21
I20250112 21:06:52.156405 3650 catalog_manager.cc:4872] TS 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): tablet 3c95203cf74b4b90867a9a3a334dbb21 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
I20250112 21:06:52.157902 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:52.158120 16217 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:52.160066 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.160590 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.161587 5864 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20250112 21:06:52.161926 12553 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20250112 21:06:52.163772 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.174418 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.175009 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.175588 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.177088 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.184489 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.202129 5601 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 2.12, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:52.202580 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20250112 21:06:52.202893 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20250112 21:06:52.203751 16215 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.205262 5470 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.205302 3664 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 12 to 13, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.207041 3917 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6bf2e45872401f8245a586b6fe7431 with delete_type TABLET_DATA_TOMBSTONED (TS fd8fa04957924e9f83afd61f9adfb455 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:35122
I20250112 21:06:52.207155 16218 tablet_replica.cc:331] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: stopping tablet replica
I20250112 21:06:52.207219 16218 raft_consensus.cc:2238] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.207268 16218 raft_consensus.cc:2267] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:52.207442 5601 consensus_queue.cc:237] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:52.207654 16218 ts_tablet_manager.cc:1905] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.207876 5470 raft_consensus.cc:1270] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20250112 21:06:52.208061 12553 consensus_queue.cc:1035] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20250112 21:06:52.208544 12553 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.14: config changed from index 13 to 14, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:52.209300 16218 ts_tablet_manager.cc:1918] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.12
I20250112 21:06:52.209350 16218 log.cc:1198] T 2e6bf2e45872401f8245a586b6fe7431 P fd8fa04957924e9f83afd61f9adfb455: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-1/wal/wals/2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:52.209676 3649 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 2e6bf2e45872401f8245a586b6fe7431 with cas_config_opid_index 13: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:52.209792 5470 raft_consensus.cc:2949] T 2e6bf2e45872401f8245a586b6fe7431 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.14: config changed from index 13 to 14, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:52.209828 3658 catalog_manager.cc:5526] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 13 to 14, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) added. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
W20250112 21:06:52.209972 5536 consensus_peers.cc:487] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c -> Peer 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): Couldn't send request to peer 38c8e8afd3f5411490f56ebf57aee109. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e6bf2e45872401f8245a586b6fe7431. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:52.210630 3650 catalog_manager.cc:4872] TS fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): tablet 2e6bf2e45872401f8245a586b6fe7431 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:52.225900 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.226414 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.226526 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.227535 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.229863 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.236258 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.236953 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.243652 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.243783 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.248193 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.256521 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.257228 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.258579 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.264868 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.265041 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.269557 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.278638 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.279420 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.281329 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.284371 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.289170 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.292119 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.294054 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.297992 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.303982 4068 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:52.304471 4199 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:52.304697 5864 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:52.305063 13315 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:52.305244 13312 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:52.306138 13312 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.307339 4199 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.307518 3662 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 8 to 9, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.307813 5864 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.308472 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet 7249078cad3e4bc3ab2b62d8887987df with delete_type TABLET_DATA_TOMBSTONED (TS 85a6e2a0c3b94dea8a094d396cfcac4b not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:56626
I20250112 21:06:52.310629 16873 tablet_replica.cc:331] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: stopping tablet replica
I20250112 21:06:52.310701 16873 raft_consensus.cc:2238] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.310779 16873 raft_consensus.cc:2267] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:52.311048 16873 ts_tablet_manager.cc:1905] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.312582 16873 ts_tablet_manager.cc:1918] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20250112 21:06:52.312642 16873 log.cc:1198] T 7249078cad3e4bc3ab2b62d8887987df P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:52.313035 3651 catalog_manager.cc:4872] TS 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): tablet 7249078cad3e4bc3ab2b62d8887987df (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:52.317742 3936 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:52.318540 4068 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20250112 21:06:52.318791 5470 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20250112 21:06:52.318802 15066 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:52.319005 15066 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:52.319833 15066 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.320022 4068 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.320904 3662 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 10 to 11, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.321954 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 with delete_type TABLET_DATA_TOMBSTONED (TS 85a6e2a0c3b94dea8a094d396cfcac4b not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:56626
I20250112 21:06:52.322053 16873 tablet_replica.cc:331] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: stopping tablet replica
I20250112 21:06:52.322113 16873 raft_consensus.cc:2238] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.322191 16873 raft_consensus.cc:2267] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:52.322397 16873 ts_tablet_manager.cc:1905] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.323746 16873 ts_tablet_manager.cc:1918] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.10
I20250112 21:06:52.323805 16873 log.cc:1198] T 0d2ae40fed4b41218aa58c523d7d9785 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:52.323765 5470 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.324210 3651 catalog_manager.cc:4872] TS 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): tablet 0d2ae40fed4b41218aa58c523d7d9785 (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:52.328014 5732 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:52.328482 3807 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20250112 21:06:52.328701 14990 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:52.330348 14990 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:52.331032 3807 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:52.332274 3662 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 10 to 11, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:52.333266 5732 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:52.333523 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 87835ddc19954987b0f17f6aa0370c10 with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:52.333609 3807 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
W20250112 21:06:52.333709 5668 consensus_peers.cc:487] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b -> Peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): Couldn't send request to peer 6f7f47d877a8407185013dba2959fddd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87835ddc19954987b0f17f6aa0370c10. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:52.333786 16929 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:52.334028 16932 tablet_replica.cc:331] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:52.334151 16932 raft_consensus.cc:2238] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.334235 16932 raft_consensus.cc:2267] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:52.334501 16932 ts_tablet_manager.cc:1905] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.334582 14990 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.335448 3651 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 87835ddc19954987b0f17f6aa0370c10 with cas_config_opid_index 11: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:52.335721 3807 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.335904 3657 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 11 to 12, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) added. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } }
I20250112 21:06:52.337688 16932 ts_tablet_manager.cc:1918] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.10
I20250112 21:06:52.337745 16932 log.cc:1198] T 87835ddc19954987b0f17f6aa0370c10 P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:52.340243 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 87835ddc19954987b0f17f6aa0370c10 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:52.360867 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.369571 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.370076 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.370474 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.380968 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.384397 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.389406 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.389988 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.391647 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.393463 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.399942 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.400136 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.403429 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.405225 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.408862 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.414192 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.442521 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.445886 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.449451 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.450659 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.451242 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.456101 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.456506 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.456718 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.461802 5732 consensus_queue.cc:237] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:52.462338 5470 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:52.462505 3807 raft_consensus.cc:1270] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:52.462549 16936 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:52.462731 16936 consensus_queue.cc:1035] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:52.465941 16936 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:52.467079 3807 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:52.467211 3664 catalog_manager.cc:5526] T 345f4bf45767475ca481525f3d26a34a P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 9 to 10, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:52.467633 5470 raft_consensus.cc:2949] T 345f4bf45767475ca481525f3d26a34a P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:52.468050 4178 tablet_service.cc:1514] Processing DeleteTablet for tablet 345f4bf45767475ca481525f3d26a34a with delete_type TABLET_DATA_TOMBSTONED (TS 1d1923072e1c45778fa5afdcfda8144d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58274
I20250112 21:06:52.468145 16932 tablet_replica.cc:331] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: stopping tablet replica
I20250112 21:06:52.468199 16932 raft_consensus.cc:2238] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.468287 16932 raft_consensus.cc:2267] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:52.468521 16932 ts_tablet_manager.cc:1905] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.470199 16932 ts_tablet_manager.cc:1918] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20250112 21:06:52.470257 16932 log.cc:1198] T 345f4bf45767475ca481525f3d26a34a P 1d1923072e1c45778fa5afdcfda8144d: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-3/wal/wals/345f4bf45767475ca481525f3d26a34a
I20250112 21:06:52.470606 3649 catalog_manager.cc:4872] TS 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): tablet 345f4bf45767475ca481525f3d26a34a (table rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]) successfully deleted
I20250112 21:06:52.485388 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.492406 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.494366 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.494611 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.494870 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.494885 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.502926 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.510284 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.533316 5864 raft_consensus.cc:1212] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.10->[2.11-2.11] Dedup: 2.11->[]
I20250112 21:06:52.536697 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.541868 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.542315 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.548138 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.549335 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.551942 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.555862 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.560150 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.563705 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.571086 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.571638 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.579704 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.583696 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.588464 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.590413 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.591634 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.593499 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.602233 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.603084 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.606395 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.607317 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.611820 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.612197 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.612560 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.622568 3936 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:52.623116 4068 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20250112 21:06:52.623301 15066 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:52.623503 5470 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
W20250112 21:06:52.623594 3871 consensus_peers.cc:487] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 -> Peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): Couldn't send request to peer 1d1923072e1c45778fa5afdcfda8144d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d2ae40fed4b41218aa58c523d7d9785. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:52.623963 15066 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:52.625267 15066 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.626556 4068 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.626586 3656 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 11 to 12, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.627861 5470 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:52.636381 17288 ts_tablet_manager.cc:927] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: Initiating tablet copy from peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:52.636682 17288 tablet_copy_client.cc:323] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.70:43655
I20250112 21:06:52.636940 5622 tablet_copy_service.cc:140] P 67a5a7038ced40ddb6f45fdbfd99825c: Received BeginTabletCopySession request for tablet 2e6bf2e45872401f8245a586b6fe7431 from peer 38c8e8afd3f5411490f56ebf57aee109 ({username='slave'} at 127.26.82.72:52587)
I20250112 21:06:52.636999 5622 tablet_copy_service.cc:161] P 67a5a7038ced40ddb6f45fdbfd99825c: Beginning new tablet copy session on tablet 2e6bf2e45872401f8245a586b6fe7431 from peer 38c8e8afd3f5411490f56ebf57aee109 at {username='slave'} at 127.26.82.72:52587: session id = 38c8e8afd3f5411490f56ebf57aee109-2e6bf2e45872401f8245a586b6fe7431
I20250112 21:06:52.637571 5622 tablet_copy_source_session.cc:215] T 2e6bf2e45872401f8245a586b6fe7431 P 67a5a7038ced40ddb6f45fdbfd99825c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:52.637918 17288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e6bf2e45872401f8245a586b6fe7431. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:52.639194 17288 tablet_copy_client.cc:806] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:52.639298 17288 tablet_copy_client.cc:670] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:52.639978 17288 tablet_copy_client.cc:538] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:52.640488 17288 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap starting.
I20250112 21:06:52.645105 17288 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:52.645313 17288 tablet_bootstrap.cc:492] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: Bootstrap complete.
I20250112 21:06:52.645416 17288 ts_tablet_manager.cc:1397] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250112 21:06:52.645512 17288 raft_consensus.cc:357] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:52.645572 17288 raft_consensus.cc:738] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c8e8afd3f5411490f56ebf57aee109, State: Initialized, Role: FOLLOWER
I20250112 21:06:52.645630 17288 consensus_queue.cc:260] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:52.645771 17288 ts_tablet_manager.cc:1428] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:52.646132 5622 tablet_copy_service.cc:342] P 67a5a7038ced40ddb6f45fdbfd99825c: Request end of tablet copy session 38c8e8afd3f5411490f56ebf57aee109-2e6bf2e45872401f8245a586b6fe7431 received from {username='slave'} at 127.26.82.72:52587
I20250112 21:06:52.646188 5622 tablet_copy_service.cc:434] P 67a5a7038ced40ddb6f45fdbfd99825c: ending tablet copy session 38c8e8afd3f5411490f56ebf57aee109-2e6bf2e45872401f8245a586b6fe7431 on tablet 2e6bf2e45872401f8245a586b6fe7431 with peer 38c8e8afd3f5411490f56ebf57aee109
I20250112 21:06:52.647914 4068 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:52.648404 4199 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
W20250112 21:06:52.648442 4003 consensus_peers.cc:487] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 -> Peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181): Couldn't send request to peer fd8fa04957924e9f83afd61f9adfb455. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7249078cad3e4bc3ab2b62d8887987df. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:52.648571 5864 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20250112 21:06:52.648598 13312 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:52.649073 13312 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:52.650243 13312 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.650432 5864 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.651722 4199 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.652051 3663 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 9 to 10, VOTER fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66) added. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:52.686057 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.690574 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.691147 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.691707 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.693084 17397 ts_tablet_manager.cc:927] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:52.696309 17397 tablet_copy_client.cc:323] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:52.696583 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 2565f97e9cc54700ab2ce39aa7a546f8 ({username='slave'} at 127.26.82.67:58023)
I20250112 21:06:52.696601 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.696633 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet 102b9dd051ff4a2a8da8ff4692e56256 from peer 2565f97e9cc54700ab2ce39aa7a546f8 at {username='slave'} at 127.26.82.67:58023: session id = 2565f97e9cc54700ab2ce39aa7a546f8-102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:52.697094 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.697157 4220 tablet_copy_source_session.cc:215] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:52.697603 17397 ts_tablet_manager.cc:1905] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:52.699280 17397 ts_tablet_manager.cc:1918] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 2.9
I20250112 21:06:52.699366 17397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b9dd051ff4a2a8da8ff4692e56256. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:52.700047 17397 tablet_copy_client.cc:806] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:52.700134 17397 tablet_copy_client.cc:670] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:52.701407 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.703446 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.705242 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.706995 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.707648 4068 raft_consensus.cc:1212] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20250112 21:06:52.711197 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.714357 5732 raft_consensus.cc:1212] T dd01159cf53642739a8a1dbcacbc933d P 85a6e2a0c3b94dea8a094d396cfcac4b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20250112 21:06:52.717336 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.721227 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.723333 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.725828 17397 tablet_copy_client.cc:538] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:52.726689 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.726809 17397 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:52.728365 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.739104 17397 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:52.739334 17397 tablet_bootstrap.cc:492] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap complete.
I20250112 21:06:52.739630 17397 ts_tablet_manager.cc:1397] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20250112 21:06:52.739780 17397 raft_consensus.cc:357] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:52.739871 17397 raft_consensus.cc:738] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:52.739935 17397 consensus_queue.cc:260] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:52.740463 17397 ts_tablet_manager.cc:1428] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:52.740953 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-102b9dd051ff4a2a8da8ff4692e56256 received from {username='slave'} at 127.26.82.67:58023
I20250112 21:06:52.741027 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-102b9dd051ff4a2a8da8ff4692e56256 on tablet 102b9dd051ff4a2a8da8ff4692e56256 with peer 2565f97e9cc54700ab2ce39aa7a546f8
I20250112 21:06:52.750490 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.750584 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.758284 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.764149 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.764647 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.767999 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.768617 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.769178 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.794958 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.796531 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.797021 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.806519 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.807174 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.807498 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.814509 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.816057 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.821413 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.826375 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.827643 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.830789 17509 ts_tablet_manager.cc:927] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: Initiating tablet copy from peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:52.831153 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.832312 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.835075 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.832151 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.837071 17509 tablet_copy_client.cc:323] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.71:37889
I20250112 21:06:52.846928 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.848174 5753 tablet_copy_service.cc:140] P 85a6e2a0c3b94dea8a094d396cfcac4b: Received BeginTabletCopySession request for tablet 87835ddc19954987b0f17f6aa0370c10 from peer 6f7f47d877a8407185013dba2959fddd ({username='slave'} at 127.26.82.69:57705)
I20250112 21:06:52.848238 5753 tablet_copy_service.cc:161] P 85a6e2a0c3b94dea8a094d396cfcac4b: Beginning new tablet copy session on tablet 87835ddc19954987b0f17f6aa0370c10 from peer 6f7f47d877a8407185013dba2959fddd at {username='slave'} at 127.26.82.69:57705: session id = 6f7f47d877a8407185013dba2959fddd-87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:52.848588 5753 tablet_copy_source_session.cc:215] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:52.849012 17509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:52.850327 17509 tablet_copy_client.cc:806] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:52.850512 17509 tablet_copy_client.cc:670] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:52.851182 17509 tablet_copy_client.cc:538] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:52.851955 17509 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: Bootstrap starting.
I20250112 21:06:52.855064 5601 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:52.855710 3807 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:52.855726 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20250112 21:06:52.855903 16215 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:52.856082 16215 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:52.857048 17509 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:52.857111 16215 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:52.857280 17509 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: Bootstrap complete.
I20250112 21:06:52.857280 3807 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:52.857400 17509 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20250112 21:06:52.857506 17509 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:52.857584 17509 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Initialized, Role: FOLLOWER
I20250112 21:06:52.857627 17509 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 2.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } }
I20250112 21:06:52.857724 17509 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:52.858603 5753 tablet_copy_service.cc:342] P 85a6e2a0c3b94dea8a094d396cfcac4b: Request end of tablet copy session 6f7f47d877a8407185013dba2959fddd-87835ddc19954987b0f17f6aa0370c10 received from {username='slave'} at 127.26.82.69:57705
I20250112 21:06:52.858664 5753 tablet_copy_service.cc:434] P 85a6e2a0c3b94dea8a094d396cfcac4b: ending tablet copy session 6f7f47d877a8407185013dba2959fddd-87835ddc19954987b0f17f6aa0370c10 on tablet 87835ddc19954987b0f17f6aa0370c10 with peer 6f7f47d877a8407185013dba2959fddd
I20250112 21:06:52.858650 3657 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 6 to 8, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:52.859097 3936 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:52.860061 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 8880a7467c6a44da96f98c3cc368c9f0 with delete_type TABLET_DATA_TOMBSTONED (TS 38c8e8afd3f5411490f56ebf57aee109 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:41748
I20250112 21:06:52.860244 17530 tablet_replica.cc:331] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: stopping tablet replica
I20250112 21:06:52.860399 17530 raft_consensus.cc:2238] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.860496 17530 raft_consensus.cc:2267] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:52.860800 17530 ts_tablet_manager.cc:1905] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.862247 17530 ts_tablet_manager.cc:1918] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20250112 21:06:52.862296 17530 log.cc:1198] T 8880a7467c6a44da96f98c3cc368c9f0 P 38c8e8afd3f5411490f56ebf57aee109: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/wals/8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:52.862599 3650 catalog_manager.cc:4872] TS 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): tablet 8880a7467c6a44da96f98c3cc368c9f0 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:52.897693 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.897810 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.898710 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.900157 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.900585 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.903498 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.912374 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.913277 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.932544 5601 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:52.932998 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:52.933213 17529 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:52.933868 17529 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:52.933971 3936 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:52.935146 3662 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:52.936196 3787 tablet_service.cc:1514] Processing DeleteTablet for tablet 8880a7467c6a44da96f98c3cc368c9f0 with delete_type TABLET_DATA_TOMBSTONED (TS bd4e9b42a6de4ff7a2d954e2be23c606 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:60766
I20250112 21:06:52.936220 5601 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:52.936393 17588 tablet_replica.cc:331] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: stopping tablet replica
I20250112 21:06:52.936546 17588 raft_consensus.cc:2238] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:52.936553 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:52.936643 17588 raft_consensus.cc:2267] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 21:06:52.936702 5537 consensus_peers.cc:487] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c -> Peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): Couldn't send request to peer 85a6e2a0c3b94dea8a094d396cfcac4b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8880a7467c6a44da96f98c3cc368c9f0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:52.937098 17528 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:52.937163 17588 ts_tablet_manager.cc:1905] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:52.937743 17529 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:52.937882 3936 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:52.938679 3649 catalog_manager.cc:5039] ChangeConfig:ADD_PEER:VOTER RPC for tablet 8880a7467c6a44da96f98c3cc368c9f0 with cas_config_opid_index 9: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20250112 21:06:52.938753 3660 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 9 to 10, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) added. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } }
I20250112 21:06:52.938869 17588 ts_tablet_manager.cc:1918] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.8
I20250112 21:06:52.938973 17588 log.cc:1198] T 8880a7467c6a44da96f98c3cc368c9f0 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-0/wal/wals/8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:52.939327 3650 catalog_manager.cc:4872] TS bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): tablet 8880a7467c6a44da96f98c3cc368c9f0 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:52.966704 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:52.970172 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.971376 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:52.971582 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.972646 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:52.973279 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:52.984179 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:52.989761 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.010084 3807 consensus_queue.cc:237] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:53.010579 5864 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:53.010721 4068 raft_consensus.cc:1270] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer bd4e9b42a6de4ff7a2d954e2be23c606: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:53.011369 17678 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:53.011461 15936 consensus_queue.cc:1035] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:53.012337 15936 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.012449 4068 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.013495 5864 raft_consensus.cc:2949] T dd9176ff75364d81a5b3bdd0e7e10673 P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.014107 3661 catalog_manager.cc:5526] T dd9176ff75364d81a5b3bdd0e7e10673 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 8 to 9, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New cstate: current_term: 1 leader_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.016080 5582 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9176ff75364d81a5b3bdd0e7e10673 with delete_type TABLET_DATA_TOMBSTONED (TS 67a5a7038ced40ddb6f45fdbfd99825c not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:49494
I20250112 21:06:53.016482 17696 tablet_replica.cc:331] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: stopping tablet replica
I20250112 21:06:53.016600 17696 raft_consensus.cc:2238] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:53.016710 17696 raft_consensus.cc:2267] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:53.016889 17696 ts_tablet_manager.cc:1905] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:53.018569 17696 ts_tablet_manager.cc:1918] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20250112 21:06:53.018627 17696 log.cc:1198] T dd9176ff75364d81a5b3bdd0e7e10673 P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/wals/dd9176ff75364d81a5b3bdd0e7e10673
I20250112 21:06:53.019032 3649 catalog_manager.cc:4872] TS 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): tablet dd9176ff75364d81a5b3bdd0e7e10673 (table rebalance_test_table_0 [id=cef690fd88994bfb9c37221904a1eaef]) successfully deleted
I20250112 21:06:53.020735 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.025255 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.026113 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.026942 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.029577 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.034034 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.034739 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.044567 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.077117 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.079789 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.087391 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.089072 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.089447 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.091145 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.093130 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.105309 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.125002 17824 ts_tablet_manager.cc:927] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: Initiating tablet copy from peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119)
I20250112 21:06:53.125365 17824 tablet_copy_client.cc:323] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.67:33119
I20250112 21:06:53.125381 5864 raft_consensus.cc:1212] T 2e6bf2e45872401f8245a586b6fe7431 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.13->[2.14-2.14] Dedup: 2.14->[]
I20250112 21:06:53.125547 4089 tablet_copy_service.cc:140] P 2565f97e9cc54700ab2ce39aa7a546f8: Received BeginTabletCopySession request for tablet 7249078cad3e4bc3ab2b62d8887987df from peer fd8fa04957924e9f83afd61f9adfb455 ({username='slave'} at 127.26.82.66:38619)
I20250112 21:06:53.125600 4089 tablet_copy_service.cc:161] P 2565f97e9cc54700ab2ce39aa7a546f8: Beginning new tablet copy session on tablet 7249078cad3e4bc3ab2b62d8887987df from peer fd8fa04957924e9f83afd61f9adfb455 at {username='slave'} at 127.26.82.66:38619: session id = fd8fa04957924e9f83afd61f9adfb455-7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:53.126164 4089 tablet_copy_source_session.cc:215] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:53.126397 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.127100 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.132827 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.138329 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.138466 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.139237 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.139999 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.142951 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.148912 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.149149 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.150095 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.153146 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.155534 17824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7249078cad3e4bc3ab2b62d8887987df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:53.157101 17824 tablet_copy_client.cc:806] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:53.157219 17824 tablet_copy_client.cc:670] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:53.158248 17824 tablet_copy_client.cc:538] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:53.159047 17824 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: Bootstrap starting.
I20250112 21:06:53.159458 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.160167 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.164052 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.167352 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.169219 17824 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:53.169422 17824 tablet_bootstrap.cc:492] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: Bootstrap complete.
I20250112 21:06:53.169512 17824 ts_tablet_manager.cc:1397] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20250112 21:06:53.169605 17824 raft_consensus.cc:357] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:53.169672 17824 raft_consensus.cc:738] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8fa04957924e9f83afd61f9adfb455, State: Initialized, Role: FOLLOWER
I20250112 21:06:53.169725 17824 consensus_queue.cc:260] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:53.169894 17824 ts_tablet_manager.cc:1428] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250112 21:06:53.170130 4089 tablet_copy_service.cc:342] P 2565f97e9cc54700ab2ce39aa7a546f8: Request end of tablet copy session fd8fa04957924e9f83afd61f9adfb455-7249078cad3e4bc3ab2b62d8887987df received from {username='slave'} at 127.26.82.66:38619
I20250112 21:06:53.170179 4089 tablet_copy_service.cc:434] P 2565f97e9cc54700ab2ce39aa7a546f8: ending tablet copy session fd8fa04957924e9f83afd61f9adfb455-7249078cad3e4bc3ab2b62d8887987df on tablet 7249078cad3e4bc3ab2b62d8887987df with peer fd8fa04957924e9f83afd61f9adfb455
I20250112 21:06:53.175168 5471 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } }
I20250112 21:06:53.175704 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:53.175719 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:53.176025 16015 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:53.176213 17843 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:53.177026 16015 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:53.178136 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:53.179199 3665 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 11 to 12, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:53.180191 4049 tablet_service.cc:1514] Processing DeleteTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e with delete_type TABLET_DATA_TOMBSTONED (TS 2565f97e9cc54700ab2ce39aa7a546f8 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:47584
I20250112 21:06:53.180469 17850 tablet_replica.cc:331] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: stopping tablet replica
I20250112 21:06:53.180563 17850 raft_consensus.cc:2238] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:53.180689 17850 raft_consensus.cc:2267] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:53.180940 17850 ts_tablet_manager.cc:1905] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:53.183104 5864 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } }
I20250112 21:06:53.183344 17850 ts_tablet_manager.cc:1918] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.11
I20250112 21:06:53.183396 17850 log.cc:1198] T de6b6ac6eeff431b9ac8cce47ea3a59e P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-2/wal/wals/de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:53.183780 3649 catalog_manager.cc:4872] TS 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): tablet de6b6ac6eeff431b9ac8cce47ea3a59e (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:53.194595 17878 ts_tablet_manager.cc:927] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: Initiating tablet copy from peer fd8fa04957924e9f83afd61f9adfb455 (127.26.82.66:42181)
I20250112 21:06:53.194738 17878 tablet_copy_client.cc:323] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.66:42181
I20250112 21:06:53.195827 3958 tablet_copy_service.cc:140] P fd8fa04957924e9f83afd61f9adfb455: Received BeginTabletCopySession request for tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 1d1923072e1c45778fa5afdcfda8144d ({username='slave'} at 127.26.82.68:40971)
I20250112 21:06:53.195889 3958 tablet_copy_service.cc:161] P fd8fa04957924e9f83afd61f9adfb455: Beginning new tablet copy session on tablet 0d2ae40fed4b41218aa58c523d7d9785 from peer 1d1923072e1c45778fa5afdcfda8144d at {username='slave'} at 127.26.82.68:40971: session id = 1d1923072e1c45778fa5afdcfda8144d-0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:53.196372 3958 tablet_copy_source_session.cc:215] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:53.196825 17878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d2ae40fed4b41218aa58c523d7d9785. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:53.198257 17878 tablet_copy_client.cc:806] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:53.198365 17878 tablet_copy_client.cc:670] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:53.198961 17878 tablet_copy_client.cc:538] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:53.199666 17878 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:53.204409 17878 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:53.204648 17878 tablet_bootstrap.cc:492] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap complete.
I20250112 21:06:53.204778 17878 ts_tablet_manager.cc:1397] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.004s
I20250112 21:06:53.204912 17878 raft_consensus.cc:357] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:53.205008 17878 raft_consensus.cc:738] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:53.205081 17878 consensus_queue.cc:260] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:53.205251 17878 ts_tablet_manager.cc:1428] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:53.205435 3958 tablet_copy_service.cc:342] P fd8fa04957924e9f83afd61f9adfb455: Request end of tablet copy session 1d1923072e1c45778fa5afdcfda8144d-0d2ae40fed4b41218aa58c523d7d9785 received from {username='slave'} at 127.26.82.68:40971
I20250112 21:06:53.205493 3958 tablet_copy_service.cc:434] P fd8fa04957924e9f83afd61f9adfb455: ending tablet copy session 1d1923072e1c45778fa5afdcfda8144d-0d2ae40fed4b41218aa58c523d7d9785 on tablet 0d2ae40fed4b41218aa58c523d7d9785 with peer 1d1923072e1c45778fa5afdcfda8144d
I20250112 21:06:53.211853 4068 raft_consensus.cc:1212] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.10->[2.11-2.11] Dedup: 2.11->[]
I20250112 21:06:53.222370 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.222594 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.223614 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.225137 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.232244 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.233263 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.235270 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.241499 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.248350 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.254386 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.257066 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.257162 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.257632 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.262988 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.264144 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.267328 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.300151 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.304555 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.307106 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.312019 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.312794 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.313014 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.313191 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.320725 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.340580 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.342069 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.342556 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.343258 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.347821 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.353432 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.356417 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.360769 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.360849 5471 raft_consensus.cc:1212] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.11->[2.12-2.12] Dedup: 2.12->[]
I20250112 21:06:53.402428 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.403374 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.405054 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.407100 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.411052 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.411458 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.412163 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.419788 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.434351 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.435643 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.437175 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.440826 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.445736 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.445974 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.446703 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.450999 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.470635 4199 consensus_queue.cc:237] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:53.471134 5864 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:53.471143 4068 raft_consensus.cc:1270] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:53.471392 18168 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:53.471477 6680 consensus_queue.cc:1035] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:53.472375 4068 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.472254 6680 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 1d1923072e1c45778fa5afdcfda8144d [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.472936 5864 raft_consensus.cc:2949] T 102b9dd051ff4a2a8da8ff4692e56256 P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.474058 3661 catalog_manager.cc:5526] T 102b9dd051ff4a2a8da8ff4692e56256 P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 11 to 12, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) evicted. New cstate: current_term: 2 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:53.474946 18171 ts_tablet_manager.cc:927] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Initiating tablet copy from peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:53.475093 18171 tablet_copy_client.cc:323] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.70:43655
I20250112 21:06:53.475242 5582 tablet_service.cc:1514] Processing DeleteTablet for tablet 102b9dd051ff4a2a8da8ff4692e56256 with delete_type TABLET_DATA_TOMBSTONED (TS 67a5a7038ced40ddb6f45fdbfd99825c not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:49494
I20250112 21:06:53.475353 17696 tablet_replica.cc:331] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: stopping tablet replica
I20250112 21:06:53.475406 17696 raft_consensus.cc:2238] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:53.475452 17696 raft_consensus.cc:2267] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:53.475626 17696 ts_tablet_manager.cc:1905] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:53.476835 17696 ts_tablet_manager.cc:1918] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.11
I20250112 21:06:53.476881 17696 log.cc:1198] T 102b9dd051ff4a2a8da8ff4692e56256 P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-5/wal/wals/102b9dd051ff4a2a8da8ff4692e56256
I20250112 21:06:53.477222 3649 catalog_manager.cc:4872] TS 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): tablet 102b9dd051ff4a2a8da8ff4692e56256 (table rebalance_test_table_5 [id=dfd586c6ebfe48f39efc00f1bd759d08]) successfully deleted
I20250112 21:06:53.481312 5622 tablet_copy_service.cc:140] P 67a5a7038ced40ddb6f45fdbfd99825c: Received BeginTabletCopySession request for tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 85a6e2a0c3b94dea8a094d396cfcac4b ({username='slave'} at 127.26.82.71:58743)
I20250112 21:06:53.481370 5622 tablet_copy_service.cc:161] P 67a5a7038ced40ddb6f45fdbfd99825c: Beginning new tablet copy session on tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 85a6e2a0c3b94dea8a094d396cfcac4b at {username='slave'} at 127.26.82.71:58743: session id = 85a6e2a0c3b94dea8a094d396cfcac4b-8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:53.481808 5622 tablet_copy_source_session.cc:215] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:53.482335 18171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8880a7467c6a44da96f98c3cc368c9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:53.483901 18171 tablet_copy_client.cc:806] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:53.484026 18171 tablet_copy_client.cc:670] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:53.484836 18171 tablet_copy_client.cc:538] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:53.485808 18171 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap starting.
I20250112 21:06:53.490762 18171 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:53.490998 18171 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Bootstrap complete.
I20250112 21:06:53.491106 18171 ts_tablet_manager.cc:1397] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20250112 21:06:53.491230 18171 raft_consensus.cc:357] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:53.491299 18171 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Initialized, Role: FOLLOWER
I20250112 21:06:53.491356 18171 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } }
I20250112 21:06:53.491468 18171 ts_tablet_manager.cc:1428] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:53.491698 5622 tablet_copy_service.cc:342] P 67a5a7038ced40ddb6f45fdbfd99825c: Request end of tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-8880a7467c6a44da96f98c3cc368c9f0 received from {username='slave'} at 127.26.82.71:58743
I20250112 21:06:53.491741 5622 tablet_copy_service.cc:434] P 67a5a7038ced40ddb6f45fdbfd99825c: ending tablet copy session 85a6e2a0c3b94dea8a094d396cfcac4b-8880a7467c6a44da96f98c3cc368c9f0 on tablet 8880a7467c6a44da96f98c3cc368c9f0 with peer 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:53.492313 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.492448 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.497251 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.497635 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.503019 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.506198 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.510418 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.513573 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.565580 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.566046 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.566319 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.566766 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.573586 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.574100 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.577739 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.580487 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.597949 3936 raft_consensus.cc:1212] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.9->[1.10-1.10] Dedup: 1.10->[]
I20250112 21:06:53.657418 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.660463 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.660528 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.661484 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.664222 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.672369 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.674083 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.677701 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.697829 4068 consensus_queue.cc:237] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } }
I20250112 21:06:53.698446 3936 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20250112 21:06:53.698446 4199 raft_consensus.cc:1270] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer 2565f97e9cc54700ab2ce39aa7a546f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20250112 21:06:53.698648 18322 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:53.698745 12524 consensus_queue.cc:1035] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:53.699465 12524 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:53.700322 3936 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P fd8fa04957924e9f83afd61f9adfb455 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:53.701503 3661 catalog_manager.cc:5526] T 7249078cad3e4bc3ab2b62d8887987df P 2565f97e9cc54700ab2ce39aa7a546f8 reported cstate change: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New cstate: current_term: 1 leader_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:53.702071 4199 raft_consensus.cc:2949] T 7249078cad3e4bc3ab2b62d8887987df P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, VOTER 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } }
I20250112 21:06:53.702395 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 7249078cad3e4bc3ab2b62d8887987df with delete_type TABLET_DATA_TOMBSTONED (TS 38c8e8afd3f5411490f56ebf57aee109 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:41748
I20250112 21:06:53.702569 18326 tablet_replica.cc:331] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: stopping tablet replica
I20250112 21:06:53.702641 18326 raft_consensus.cc:2238] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:53.702731 18326 raft_consensus.cc:2267] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:53.702908 18326 ts_tablet_manager.cc:1905] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:53.704610 18326 ts_tablet_manager.cc:1918] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.10
I20250112 21:06:53.704711 18326 log.cc:1198] T 7249078cad3e4bc3ab2b62d8887987df P 38c8e8afd3f5411490f56ebf57aee109: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-7/wal/wals/7249078cad3e4bc3ab2b62d8887987df
I20250112 21:06:53.705159 3650 catalog_manager.cc:4872] TS 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429): tablet 7249078cad3e4bc3ab2b62d8887987df (table rebalance_test_table_9 [id=d647cf606b06442ebf35d3ebc20f7721]) successfully deleted
I20250112 21:06:53.715677 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.717341 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.721678 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.721796 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.726094 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.726339 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.727097 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.727799 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.755872 5732 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 2.12, Last appended by leader: 7, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:53.756626 3807 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
W20250112 21:06:53.756783 5666 consensus_peers.cc:487] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:53.756889 5471 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Refusing update from remote peer 85a6e2a0c3b94dea8a094d396cfcac4b: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20250112 21:06:53.756984 16929 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20250112 21:06:53.757094 18387 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20250112 21:06:53.758637 18388 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:53.761379 3807 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:53.761559 3661 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b reported cstate change: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 2 leader_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:53.761894 5471 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:53.770574 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.771363 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.773319 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.775229 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.780149 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.781554 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.789497 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.789978 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.792361 4199 raft_consensus.cc:1212] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20250112 21:06:53.797541 5471 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 2.12, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:53.798851 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20250112 21:06:53.798913 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Refusing update from remote peer 6f7f47d877a8407185013dba2959fddd: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20250112 21:06:53.799101 18434 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20250112 21:06:53.799238 18434 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20250112 21:06:53.800348 18435 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:53.801815 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:53.801815 3660 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd reported cstate change: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New cstate: current_term: 2 leader_uuid: "6f7f47d877a8407185013dba2959fddd" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
W20250112 21:06:53.802209 5404 consensus_peers.cc:487] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd -> Peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655): Couldn't send request to peer 67a5a7038ced40ddb6f45fdbfd99825c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de6b6ac6eeff431b9ac8cce47ea3a59e. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:53.803375 5864 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:53.836124 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.841378 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.847200 5580 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.847301 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.849006 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.852569 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.852909 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.853396 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.861071 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.861521 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.867594 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.869357 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.869496 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.869689 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.870235 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.873458 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.926328 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.929620 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.937630 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.939301 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.940227 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.941305 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.946872 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.947538 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.948312 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.952728 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.953094 5732 raft_consensus.cc:1212] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.9->[2.10-2.10] Dedup: 2.10->[]
I20250112 21:06:53.956678 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.956800 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:53.957048 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.963650 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.964625 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.968245 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.984045 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:53.987080 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:53.991362 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.994876 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:53.997066 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:53.999936 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.005592 3936 consensus_queue.cc:237] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:54.007834 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.007985 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.010170 4199 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20250112 21:06:54.010387 4068 raft_consensus.cc:1270] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer fd8fa04957924e9f83afd61f9adfb455: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20250112 21:06:54.010385 18683 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20250112 21:06:54.010607 18683 consensus_queue.cc:1035] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20250112 21:06:54.011096 18683 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.012552 4199 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 1d1923072e1c45778fa5afdcfda8144d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.012701 3665 catalog_manager.cc:5526] T 0d2ae40fed4b41218aa58c523d7d9785 P fd8fa04957924e9f83afd61f9adfb455 reported cstate change: config changed from index 12 to 13, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New cstate: current_term: 1 leader_uuid: "fd8fa04957924e9f83afd61f9adfb455" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.014312 5451 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d2ae40fed4b41218aa58c523d7d9785 with delete_type TABLET_DATA_TOMBSTONED (TS 6f7f47d877a8407185013dba2959fddd not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:51422
I20250112 21:06:54.014462 4068 raft_consensus.cc:2949] T 0d2ae40fed4b41218aa58c523d7d9785 P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.020845 18693 tablet_replica.cc:331] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: stopping tablet replica
I20250112 21:06:54.020911 18693 raft_consensus.cc:2238] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:54.020960 18693 raft_consensus.cc:2267] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:54.021162 18693 ts_tablet_manager.cc:1905] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:54.022418 18693 ts_tablet_manager.cc:1918] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20250112 21:06:54.022471 18693 log.cc:1198] T 0d2ae40fed4b41218aa58c523d7d9785 P 6f7f47d877a8407185013dba2959fddd: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/wals/0d2ae40fed4b41218aa58c523d7d9785
I20250112 21:06:54.022789 3651 catalog_manager.cc:4872] TS 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): tablet 0d2ae40fed4b41218aa58c523d7d9785 (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:54.041586 5601 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
W20250112 21:06:54.042203 5535 consensus_peers.cc:487] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c -> Peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569): Couldn't send request to peer 1d1923072e1c45778fa5afdcfda8144d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8880a7467c6a44da96f98c3cc368c9f0. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:54.042210 3936 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20250112 21:06:54.042404 16215 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:54.042675 5732 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20250112 21:06:54.042876 18752 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20250112 21:06:54.043496 16215 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.043596 5732 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.044588 3661 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 10 to 11, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.044571 3936 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, VOTER 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:54.056244 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.057607 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.057801 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.059991 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.064289 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.066835 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.075242 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.075352 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.079761 4199 consensus_queue.cc:237] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
W20250112 21:06:54.080785 4133 consensus_peers.cc:487] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d -> Peer 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67:33119): Couldn't send request to peer 2565f97e9cc54700ab2ce39aa7a546f8. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:54.081128 5471 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:54.081399 18795 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:54.083998 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.085361 5864 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20250112 21:06:54.090142 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.092317 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.092458 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.092557 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.092919 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.093106 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.099191 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.093516 18795 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20250112 21:06:54.100454 18795 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:54.100543 5864 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:54.101882 3660 catalog_manager.cc:5526] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 reported cstate change: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:54.102356 5471 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, VOTER 2565f97e9cc54700ab2ce39aa7a546f8 (127.26.82.67) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:54.150233 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.152093 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.152109 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.161224 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.162022 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.163825 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.167362 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.170296 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.171569 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.174795 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.179260 18921 ts_tablet_manager.cc:927] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:54.179420 18921 tablet_copy_client.cc:323] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.69:40871
I20250112 21:06:54.179689 5485 tablet_copy_service.cc:140] P 6f7f47d877a8407185013dba2959fddd: Received BeginTabletCopySession request for tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:52321)
I20250112 21:06:54.179754 5485 tablet_copy_service.cc:161] P 6f7f47d877a8407185013dba2959fddd: Beginning new tablet copy session on tablet de6b6ac6eeff431b9ac8cce47ea3a59e from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:52321: session id = 67a5a7038ced40ddb6f45fdbfd99825c-de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:54.179844 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.180394 5485 tablet_copy_source_session.cc:215] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:54.180788 18921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de6b6ac6eeff431b9ac8cce47ea3a59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:54.182060 18921 tablet_copy_client.cc:806] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:54.182163 18921 tablet_copy_client.cc:670] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:54.183813 18921 tablet_copy_client.cc:538] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:54.184578 18921 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:54.188294 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.189981 18921 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:54.190209 18921 tablet_bootstrap.cc:492] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:54.190308 18921 ts_tablet_manager.cc:1397] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20250112 21:06:54.190409 18921 raft_consensus.cc:357] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:54.190477 18921 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:54.190826 18921 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 2.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:54.191192 18921 ts_tablet_manager.cc:1428] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250112 21:06:54.191391 5485 tablet_copy_service.cc:342] P 6f7f47d877a8407185013dba2959fddd: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-de6b6ac6eeff431b9ac8cce47ea3a59e received from {username='slave'} at 127.26.82.70:52321
I20250112 21:06:54.191430 5485 tablet_copy_service.cc:434] P 6f7f47d877a8407185013dba2959fddd: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-de6b6ac6eeff431b9ac8cce47ea3a59e on tablet de6b6ac6eeff431b9ac8cce47ea3a59e with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:54.192821 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.195780 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.197166 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.198320 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.207371 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.208881 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.215361 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.218809 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.219126 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.225512 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.226819 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.232297 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.238447 18921 ts_tablet_manager.cc:927] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Initiating tablet copy from peer 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889)
I20250112 21:06:54.238569 18921 tablet_copy_client.cc:323] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.71:37889
I20250112 21:06:54.238736 5753 tablet_copy_service.cc:140] P 85a6e2a0c3b94dea8a094d396cfcac4b: Received BeginTabletCopySession request for tablet 87835ddc19954987b0f17f6aa0370c10 from peer 67a5a7038ced40ddb6f45fdbfd99825c ({username='slave'} at 127.26.82.70:58249)
I20250112 21:06:54.238785 5753 tablet_copy_service.cc:161] P 85a6e2a0c3b94dea8a094d396cfcac4b: Beginning new tablet copy session on tablet 87835ddc19954987b0f17f6aa0370c10 from peer 67a5a7038ced40ddb6f45fdbfd99825c at {username='slave'} at 127.26.82.70:58249: session id = 67a5a7038ced40ddb6f45fdbfd99825c-87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:54.239372 5753 tablet_copy_source_session.cc:215] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:54.239706 18921 ts_tablet_manager.cc:1905] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:54.241487 18921 ts_tablet_manager.cc:1918] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20250112 21:06:54.241608 18921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87835ddc19954987b0f17f6aa0370c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:54.242280 18921 tablet_copy_client.cc:806] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:54.242437 18921 tablet_copy_client.cc:670] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:54.243361 18921 tablet_copy_client.cc:538] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:54.243849 18921 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap starting.
I20250112 21:06:54.248451 18921 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:54.248639 18921 tablet_bootstrap.cc:492] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Bootstrap complete.
I20250112 21:06:54.248754 18921 ts_tablet_manager.cc:1397] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20250112 21:06:54.248941 18921 raft_consensus.cc:357] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:54.249028 18921 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Initialized, Role: FOLLOWER
I20250112 21:06:54.249097 18921 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 2.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:54.249329 18921 ts_tablet_manager.cc:1428] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250112 21:06:54.249606 5753 tablet_copy_service.cc:342] P 85a6e2a0c3b94dea8a094d396cfcac4b: Request end of tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-87835ddc19954987b0f17f6aa0370c10 received from {username='slave'} at 127.26.82.70:58249
I20250112 21:06:54.249670 5753 tablet_copy_service.cc:434] P 85a6e2a0c3b94dea8a094d396cfcac4b: ending tablet copy session 67a5a7038ced40ddb6f45fdbfd99825c-87835ddc19954987b0f17f6aa0370c10 on tablet 87835ddc19954987b0f17f6aa0370c10 with peer 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:54.272274 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.279067 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.280498 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.280941 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.286280 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.293587 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.294723 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.296320 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.302402 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.303445 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.307518 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.308794 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.309010 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.309983 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.314005 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.316818 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.327826 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.330766 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.331933 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.339326 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.340449 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.344030 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.349226 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.353957 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.357187 5601 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:54.357749 4199 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
W20250112 21:06:54.357955 5536 consensus_peers.cc:487] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c -> Peer bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903): Couldn't send request to peer bd4e9b42a6de4ff7a2d954e2be23c606. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250112 21:06:54.358060 5732 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20250112 21:06:54.358237 18751 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:54.358340 18752 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:54.360443 18752 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:54.361923 5732 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:54.362051 4199 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:54.362125 3664 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 8 to 9, VOTER bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65) added. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:54.394490 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.396024 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.398535 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.400689 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.400928 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.403319 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.409071 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.410140 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.445003 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.445561 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.449892 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.450261 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.454142 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.455955 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.456969 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.457398 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.494880 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.496925 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.499089 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.502831 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.505391 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.509358 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.509441 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.512884 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.530474 19221 ts_tablet_manager.cc:927] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Initiating tablet copy from peer 1d1923072e1c45778fa5afdcfda8144d (127.26.82.68:33569)
I20250112 21:06:54.530627 19221 tablet_copy_client.cc:323] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.68:33569
I20250112 21:06:54.530856 4220 tablet_copy_service.cc:140] P 1d1923072e1c45778fa5afdcfda8144d: Received BeginTabletCopySession request for tablet b6ba48c8dc7e4ee485f85ee94aca256b from peer 2565f97e9cc54700ab2ce39aa7a546f8 ({username='slave'} at 127.26.82.67:58023)
I20250112 21:06:54.530913 4220 tablet_copy_service.cc:161] P 1d1923072e1c45778fa5afdcfda8144d: Beginning new tablet copy session on tablet b6ba48c8dc7e4ee485f85ee94aca256b from peer 2565f97e9cc54700ab2ce39aa7a546f8 at {username='slave'} at 127.26.82.67:58023: session id = 2565f97e9cc54700ab2ce39aa7a546f8-b6ba48c8dc7e4ee485f85ee94aca256b
I20250112 21:06:54.531291 4220 tablet_copy_source_session.cc:215] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:54.531585 19221 ts_tablet_manager.cc:1905] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:54.533452 19221 ts_tablet_manager.cc:1918] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20250112 21:06:54.533555 19221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ba48c8dc7e4ee485f85ee94aca256b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:54.534230 19221 tablet_copy_client.cc:806] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:54.534329 19221 tablet_copy_client.cc:670] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:54.534996 19221 tablet_copy_client.cc:538] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:54.535645 19221 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap starting.
I20250112 21:06:54.539945 19221 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:54.540112 19221 tablet_bootstrap.cc:492] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Bootstrap complete.
I20250112 21:06:54.540203 19221 ts_tablet_manager.cc:1397] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20250112 21:06:54.540309 19221 raft_consensus.cc:357] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:54.540382 19221 raft_consensus.cc:738] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2565f97e9cc54700ab2ce39aa7a546f8, State: Initialized, Role: FOLLOWER
I20250112 21:06:54.540426 19221 consensus_queue.cc:260] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:54.540501 19221 ts_tablet_manager.cc:1428] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:54.541079 4220 tablet_copy_service.cc:342] P 1d1923072e1c45778fa5afdcfda8144d: Request end of tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-b6ba48c8dc7e4ee485f85ee94aca256b received from {username='slave'} at 127.26.82.67:58023
I20250112 21:06:54.541136 4220 tablet_copy_service.cc:434] P 1d1923072e1c45778fa5afdcfda8144d: ending tablet copy session 2565f97e9cc54700ab2ce39aa7a546f8-b6ba48c8dc7e4ee485f85ee94aca256b on tablet b6ba48c8dc7e4ee485f85ee94aca256b with peer 2565f97e9cc54700ab2ce39aa7a546f8
I20250112 21:06:54.556643 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.557065 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.562435 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.562889 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.563220 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.563467 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.567677 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.578027 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.590698 19273 ts_tablet_manager.cc:927] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: Initiating tablet copy from peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:54.590828 19273 tablet_copy_client.cc:323] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.70:43655
I20250112 21:06:54.591187 5622 tablet_copy_service.cc:140] P 67a5a7038ced40ddb6f45fdbfd99825c: Received BeginTabletCopySession request for tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 1d1923072e1c45778fa5afdcfda8144d ({username='slave'} at 127.26.82.68:55505)
I20250112 21:06:54.591270 5622 tablet_copy_service.cc:161] P 67a5a7038ced40ddb6f45fdbfd99825c: Beginning new tablet copy session on tablet 8880a7467c6a44da96f98c3cc368c9f0 from peer 1d1923072e1c45778fa5afdcfda8144d at {username='slave'} at 127.26.82.68:55505: session id = 1d1923072e1c45778fa5afdcfda8144d-8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:54.591897 5622 tablet_copy_source_session.cc:215] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:54.592239 19273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8880a7467c6a44da96f98c3cc368c9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:54.593869 19273 tablet_copy_client.cc:806] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:54.594004 19273 tablet_copy_client.cc:670] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:54.594516 19273 tablet_copy_client.cc:538] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:54.595053 19273 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap starting.
I20250112 21:06:54.598773 19273 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:54.598984 19273 tablet_bootstrap.cc:492] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: Bootstrap complete.
I20250112 21:06:54.599076 19273 ts_tablet_manager.cc:1397] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250112 21:06:54.599186 19273 raft_consensus.cc:357] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:54.599256 19273 raft_consensus.cc:738] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1923072e1c45778fa5afdcfda8144d, State: Initialized, Role: FOLLOWER
I20250112 21:06:54.599313 19273 consensus_queue.cc:260] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:54.599402 19273 ts_tablet_manager.cc:1428] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d: Time spent starting tablet: real 0.000s user 0.003s sys 0.000s
I20250112 21:06:54.599617 5622 tablet_copy_service.cc:342] P 67a5a7038ced40ddb6f45fdbfd99825c: Request end of tablet copy session 1d1923072e1c45778fa5afdcfda8144d-8880a7467c6a44da96f98c3cc368c9f0 received from {username='slave'} at 127.26.82.68:55505
I20250112 21:06:54.599665 5622 tablet_copy_service.cc:434] P 67a5a7038ced40ddb6f45fdbfd99825c: ending tablet copy session 1d1923072e1c45778fa5afdcfda8144d-8880a7467c6a44da96f98c3cc368c9f0 on tablet 8880a7467c6a44da96f98c3cc368c9f0 with peer 1d1923072e1c45778fa5afdcfda8144d
I20250112 21:06:54.604435 5601 raft_consensus.cc:1212] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.12->[2.13-2.13] Dedup: 2.13->[]
I20250112 21:06:54.615891 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.616261 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.623437 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.627929 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.628384 5601 raft_consensus.cc:1212] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.12->[2.13-2.13] Dedup: 2.13->[]
I20250112 21:06:54.630191 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.630760 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.631927 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.642009 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.643543 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.646495 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.650791 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.652186 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.652223 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.655005 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.658308 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.658546 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.694460 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.695046 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.700009 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.701117 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.707437 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.712592 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.712924 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.715903 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.721661 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.723681 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.726606 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.730103 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.731344 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.733564 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.738773 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.739672 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.744685 19424 ts_tablet_manager.cc:927] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Initiating tablet copy from peer 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70:43655)
I20250112 21:06:54.744871 19424 tablet_copy_client.cc:323] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Beginning tablet copy session from remote peer at address 127.26.82.70:43655
I20250112 21:06:54.745061 5622 tablet_copy_service.cc:140] P 67a5a7038ced40ddb6f45fdbfd99825c: Received BeginTabletCopySession request for tablet b3b6276ae83948b6b4ea2771d9ce8355 from peer bd4e9b42a6de4ff7a2d954e2be23c606 ({username='slave'} at 127.26.82.65:54203)
I20250112 21:06:54.745127 5622 tablet_copy_service.cc:161] P 67a5a7038ced40ddb6f45fdbfd99825c: Beginning new tablet copy session on tablet b3b6276ae83948b6b4ea2771d9ce8355 from peer bd4e9b42a6de4ff7a2d954e2be23c606 at {username='slave'} at 127.26.82.65:54203: session id = bd4e9b42a6de4ff7a2d954e2be23c606-b3b6276ae83948b6b4ea2771d9ce8355
I20250112 21:06:54.745682 5622 tablet_copy_source_session.cc:215] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 21:06:54.746013 19424 ts_tablet_manager.cc:1905] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250112 21:06:54.747763 19424 ts_tablet_manager.cc:1918] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 2.7
I20250112 21:06:54.748063 19424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b6276ae83948b6b4ea2771d9ce8355. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 21:06:54.748975 19424 tablet_copy_client.cc:806] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Starting download of 0 data blocks...
I20250112 21:06:54.749087 19424 tablet_copy_client.cc:670] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Starting download of 1 WAL segments...
I20250112 21:06:54.749584 19424 tablet_copy_client.cc:538] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 21:06:54.750053 19424 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap starting.
I20250112 21:06:54.753157 19424 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 21:06:54.753310 19424 tablet_bootstrap.cc:492] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Bootstrap complete.
I20250112 21:06:54.753379 19424 ts_tablet_manager.cc:1397] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20250112 21:06:54.753481 19424 raft_consensus.cc:357] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:54.753552 19424 raft_consensus.cc:738] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4e9b42a6de4ff7a2d954e2be23c606, State: Initialized, Role: FOLLOWER
I20250112 21:06:54.753593 19424 consensus_queue.cc:260] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:54.753712 19424 ts_tablet_manager.cc:1428] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250112 21:06:54.753955 5622 tablet_copy_service.cc:342] P 67a5a7038ced40ddb6f45fdbfd99825c: Request end of tablet copy session bd4e9b42a6de4ff7a2d954e2be23c606-b3b6276ae83948b6b4ea2771d9ce8355 received from {username='slave'} at 127.26.82.65:54203
I20250112 21:06:54.754004 5622 tablet_copy_service.cc:434] P 67a5a7038ced40ddb6f45fdbfd99825c: ending tablet copy session bd4e9b42a6de4ff7a2d954e2be23c606-b3b6276ae83948b6b4ea2771d9ce8355 on tablet b3b6276ae83948b6b4ea2771d9ce8355 with peer bd4e9b42a6de4ff7a2d954e2be23c606
I20250112 21:06:54.902736 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.903834 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.908026 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.909322 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.910706 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.912705 4068 raft_consensus.cc:1212] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20250112 21:06:54.915445 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.918814 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.919620 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.929813 5471 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e"
dest_uuid: "6f7f47d877a8407185013dba2959fddd"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55000
I20250112 21:06:54.929908 5471 raft_consensus.cc:604] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Received request to transfer leadership
I20250112 21:06:54.958422 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.959697 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.964279 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.965409 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.970611 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.974885 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:54.976423 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:54.980178 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:54.992516 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.996006 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:54.997253 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:54.997422 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:55.000837 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.002303 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.012861 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.013450 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.029765 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.033849 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.035893 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.039995 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.042977 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.043617 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:55.052096 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.052559 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.070328 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.077114 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.078862 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.079337 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.080729 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.084843 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:55.087451 4199 consensus_queue.cc:237] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } }
I20250112 21:06:55.088100 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.089332 5864 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:55.089526 18803 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:55.090004 4068 raft_consensus.cc:1270] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Refusing update from remote peer 1d1923072e1c45778fa5afdcfda8144d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20250112 21:06:55.090186 18803 consensus_queue.cc:1035] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20250112 21:06:55.090500 18803 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:55.091722 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.091739 5864 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 38c8e8afd3f5411490f56ebf57aee109 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:55.091933 3665 catalog_manager.cc:5526] T b6ba48c8dc7e4ee485f85ee94aca256b P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New cstate: current_term: 1 leader_uuid: "1d1923072e1c45778fa5afdcfda8144d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:55.095525 5451 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ba48c8dc7e4ee485f85ee94aca256b with delete_type TABLET_DATA_TOMBSTONED (TS 6f7f47d877a8407185013dba2959fddd not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51422
I20250112 21:06:55.095779 19660 tablet_replica.cc:331] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: stopping tablet replica
I20250112 21:06:55.095852 19660 raft_consensus.cc:2238] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:55.095925 19660 raft_consensus.cc:2267] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:55.096133 4068 raft_consensus.cc:2949] T b6ba48c8dc7e4ee485f85ee94aca256b P 2565f97e9cc54700ab2ce39aa7a546f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "2565f97e9cc54700ab2ce39aa7a546f8" member_type: VOTER last_known_addr { host: "127.26.82.67" port: 33119 } } }
I20250112 21:06:55.096215 19660 ts_tablet_manager.cc:1905] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:55.097662 19660 ts_tablet_manager.cc:1918] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20250112 21:06:55.097709 19660 log.cc:1198] T b6ba48c8dc7e4ee485f85ee94aca256b P 6f7f47d877a8407185013dba2959fddd: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/wals/b6ba48c8dc7e4ee485f85ee94aca256b
I20250112 21:06:55.098167 3651 catalog_manager.cc:4872] TS 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): tablet b6ba48c8dc7e4ee485f85ee94aca256b (table rebalance_test_table_4 [id=f456baf32c6f4feba4f57d67d5f2dc09]) successfully deleted
I20250112 21:06:55.108495 5732 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10"
dest_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56952
I20250112 21:06:55.108584 5732 raft_consensus.cc:604] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Received request to transfer leadership
I20250112 21:06:55.131991 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.134733 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.139142 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.141568 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.144475 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:55.147509 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.152822 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.153599 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.162804 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.165403 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.166132 4199 raft_consensus.cc:1212] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.10->[2.11-2.11] Dedup: 2.11->[]
I20250112 21:06:55.168779 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.169406 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.171694 5582 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20250112 21:06:55.172380 5451 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.175804 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.179323 5713 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20250112 21:06:55.183423 18434 raft_consensus.cc:988] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: : Instructing follower 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:55.183490 18434 raft_consensus.cc:1076] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Signalling peer 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:55.183693 5602 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e"
dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
from {username='slave'} at 127.26.82.69:35909
I20250112 21:06:55.183779 5602 raft_consensus.cc:491] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:55.183810 5602 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.184579 5602 raft_consensus.cc:513] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:55.184792 5602 leader_election.cc:290] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 3 election: Requested vote from peers bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871), 38c8e8afd3f5411490f56ebf57aee109 (127.26.82.72:37429)
I20250112 21:06:55.185076 5471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "6f7f47d877a8407185013dba2959fddd"
I20250112 21:06:55.185097 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "38c8e8afd3f5411490f56ebf57aee109"
I20250112 21:06:55.185148 5471 raft_consensus.cc:3049] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Stepping down as leader of term 2
I20250112 21:06:55.185168 5864 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.185173 5471 raft_consensus.cc:738] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f7f47d877a8407185013dba2959fddd, State: Running, Role: LEADER
I20250112 21:06:55.185220 5471 consensus_queue.cc:260] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 13, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:55.185295 5471 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.185685 5864 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 3.
I20250112 21:06:55.185844 5471 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 3.
I20250112 21:06:55.185942 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de6b6ac6eeff431b9ac8cce47ea3a59e" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:55.186018 3807 raft_consensus.cc:3054] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.186012 5537 leader_election.cc:304] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 38c8e8afd3f5411490f56ebf57aee109, 67a5a7038ced40ddb6f45fdbfd99825c, 6f7f47d877a8407185013dba2959fddd; no voters:
I20250112 21:06:55.186120 19088 raft_consensus.cc:2798] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 FOLLOWER]: Leader election won for term 3
I20250112 21:06:55.186234 19088 raft_consensus.cc:695] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 LEADER]: Becoming Leader. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Running, Role: LEADER
I20250112 21:06:55.186288 19088 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 13, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:55.186760 3807 raft_consensus.cc:2463] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 3.
I20250112 21:06:55.187040 3656 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: term changed from 2 to 3, leader changed from 6f7f47d877a8407185013dba2959fddd (127.26.82.69) to 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70). New cstate: current_term: 3 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:55.225452 19749 raft_consensus.cc:988] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: : Instructing follower 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:55.225525 19749 raft_consensus.cc:1076] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Signalling peer 67a5a7038ced40ddb6f45fdbfd99825c to start an election
I20250112 21:06:55.225713 5602 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10"
dest_uuid: "67a5a7038ced40ddb6f45fdbfd99825c"
from {username='slave'} at 127.26.82.71:58743
I20250112 21:06:55.225776 5602 raft_consensus.cc:491] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 21:06:55.225795 5602 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.226333 5602 raft_consensus.cc:513] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:55.226454 5602 leader_election.cc:290] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 3 election: Requested vote from peers 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889), bd4e9b42a6de4ff7a2d954e2be23c606 (127.26.82.65:42903), 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871)
I20250112 21:06:55.226619 5732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b"
I20250112 21:06:55.226646 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606"
I20250112 21:06:55.226745 5732 raft_consensus.cc:3049] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Stepping down as leader of term 2
I20250112 21:06:55.226745 3807 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.226661 5471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87835ddc19954987b0f17f6aa0370c10" candidate_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" candidate_term: 3 candidate_status { last_received { term: 2 index: 13 } } ignore_live_leader: true dest_uuid: "6f7f47d877a8407185013dba2959fddd"
I20250112 21:06:55.226773 5732 raft_consensus.cc:738] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 85a6e2a0c3b94dea8a094d396cfcac4b, State: Running, Role: LEADER
I20250112 21:06:55.226785 5471 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.226823 5732 consensus_queue.cc:260] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 13, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:55.226909 5732 raft_consensus.cc:3054] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Advancing to term 3
I20250112 21:06:55.227355 5471 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 3.
I20250112 21:06:55.227504 3807 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 3.
I20250112 21:06:55.227634 5732 raft_consensus.cc:2463] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67a5a7038ced40ddb6f45fdbfd99825c in term 3.
I20250112 21:06:55.227744 5536 leader_election.cc:304] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 67a5a7038ced40ddb6f45fdbfd99825c, 6f7f47d877a8407185013dba2959fddd, bd4e9b42a6de4ff7a2d954e2be23c606; no voters:
I20250112 21:06:55.227840 19088 raft_consensus.cc:2798] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 FOLLOWER]: Leader election won for term 3
I20250112 21:06:55.227909 19088 raft_consensus.cc:695] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 LEADER]: Becoming Leader. State: Replica: 67a5a7038ced40ddb6f45fdbfd99825c, State: Running, Role: LEADER
I20250112 21:06:55.227978 19088 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 13, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:55.228521 3656 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: term changed from 2 to 3, leader changed from 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) to 67a5a7038ced40ddb6f45fdbfd99825c (127.26.82.70). New cstate: current_term: 3 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:55.256845 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.258136 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.259069 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.259938 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.261559 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.263981 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.271641 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.272296 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.328814 3807 raft_consensus.cc:1212] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.8->[2.9-2.9] Dedup: 2.9->[]
I20250112 21:06:55.417313 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.418495 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.421676 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.424057 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.431334 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.431641 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.432169 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.433058 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.434741 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.445469 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.446002 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.448997 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.451629 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.457206 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.459113 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.461470 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.473747 5602 consensus_queue.cc:237] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } }
I20250112 21:06:55.474226 4199 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:55.474313 3938 raft_consensus.cc:1270] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20250112 21:06:55.474470 19747 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:55.474999 19747 consensus_queue.cc:1035] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20250112 21:06:55.475240 19747 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:55.475498 4199 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:55.476577 3658 catalog_manager.cc:5526] T 8880a7467c6a44da96f98c3cc368c9f0 P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 11 to 12, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:55.477149 3938 raft_consensus.cc:2949] T 8880a7467c6a44da96f98c3cc368c9f0 P fd8fa04957924e9f83afd61f9adfb455 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "fd8fa04957924e9f83afd61f9adfb455" member_type: VOTER last_known_addr { host: "127.26.82.66" port: 42181 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } }
I20250112 21:06:55.478803 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet 8880a7467c6a44da96f98c3cc368c9f0 with delete_type TABLET_DATA_TOMBSTONED (TS 85a6e2a0c3b94dea8a094d396cfcac4b not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:56626
I20250112 21:06:55.479254 19919 tablet_replica.cc:331] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: stopping tablet replica
I20250112 21:06:55.479418 19919 raft_consensus.cc:2238] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:55.479467 19919 raft_consensus.cc:2267] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:55.479636 19919 ts_tablet_manager.cc:1905] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:55.481117 19919 ts_tablet_manager.cc:1918] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.11
I20250112 21:06:55.481174 19919 log.cc:1198] T 8880a7467c6a44da96f98c3cc368c9f0 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/8880a7467c6a44da96f98c3cc368c9f0
I20250112 21:06:55.481551 3651 catalog_manager.cc:4872] TS 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): tablet 8880a7467c6a44da96f98c3cc368c9f0 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted
I20250112 21:06:55.526865 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.529392 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.530292 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.532544 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.536221 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.537184 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.542173 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.543927 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.544474 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.544936 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.551334 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.556000 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.558084 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.563767 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.564810 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.574115 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.575049 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.576252 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.577394 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.578768 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.580111 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.587828 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.592108 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.600317 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.600854 5602 consensus_queue.cc:237] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } }
I20250112 21:06:55.601356 3807 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:55.601366 4199 raft_consensus.cc:1270] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250112 21:06:55.601562 19747 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:55.601632 19088 consensus_queue.cc:1035] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250112 21:06:55.602384 19088 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 67a5a7038ced40ddb6f45fdbfd99825c [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:55.602500 4199 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:55.602515 3807 raft_consensus.cc:2949] T b3b6276ae83948b6b4ea2771d9ce8355 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:55.603617 3665 catalog_manager.cc:5526] T b3b6276ae83948b6b4ea2771d9ce8355 P 1d1923072e1c45778fa5afdcfda8144d reported cstate change: config changed from index 9 to 10, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New cstate: current_term: 2 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } }
I20250112 21:06:55.604468 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b6276ae83948b6b4ea2771d9ce8355 with delete_type TABLET_DATA_TOMBSTONED (TS 85a6e2a0c3b94dea8a094d396cfcac4b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:56626
I20250112 21:06:55.604584 19919 tablet_replica.cc:331] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: stopping tablet replica
I20250112 21:06:55.604683 19919 raft_consensus.cc:2238] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:55.604743 19919 raft_consensus.cc:2267] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:55.604980 19919 ts_tablet_manager.cc:1905] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:55.606365 19919 ts_tablet_manager.cc:1918] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20250112 21:06:55.606426 19919 log.cc:1198] T b3b6276ae83948b6b4ea2771d9ce8355 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/b3b6276ae83948b6b4ea2771d9ce8355
I20250112 21:06:55.606812 3651 catalog_manager.cc:4872] TS 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): tablet b3b6276ae83948b6b4ea2771d9ce8355 (table rebalance_test_table_6 [id=7fdab271d7cd4decb290f197850245c2]) successfully deleted
I20250112 21:06:55.620297 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20250112 21:06:55.620494 20057 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20250112 21:06:55.631352 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20250112 21:06:55.631616 20057 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20250112 21:06:55.633487 5471 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20250112 21:06:55.633821 19747 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20250112 21:06:55.640280 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.641850 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.645805 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.646723 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.647286 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.648864 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.653421 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.653960 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.714277 5471 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20250112 21:06:55.714579 19747 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20250112 21:06:55.724375 3807 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20250112 21:06:55.724604 19747 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20250112 21:06:55.726450 5732 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20250112 21:06:55.726680 19747 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85a6e2a0c3b94dea8a094d396cfcac4b" member_type: VOTER last_known_addr { host: "127.26.82.71" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20250112 21:06:55.827538 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.833675 5913 rebalancer_tool-test.cc:1260] rebalancer run info: Runtime error: /tmp/dist-test-taskam1Qel/build/release/bin/kudu: process exited on signal 11 (core dumped)
stdout: Per-server replica distribution summary:
Statistic | Value
-----------------------+-----------
Minimum Replica Count | 0
Maximum Replica Count | 23
Average Replica Count | 11.250000
Per-table replica distribution summary:
Replica Skew | Value
--------------+----------
Minimum | 3
Maximum | 3
Average | 3.000000
rebalancing is complete: cluster is balanced (moved 25 replicas)
Per-server replica distribution summary:
Statistic | Value
-----------------------+-----------
Minimum Replica Count | 11
Maximum Replica Count | 12
Average Replica Count | 11.625000
Per-table replica distribution summary:
Replica Skew | Value
--------------+----------
Minimum | 1
Maximum | 1
Average | 1.000000
stderr: W20250112 21:06:44.014565 5918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250112 21:06:44.014752 5918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250112 21:06:44.016705 5918 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250112 21:06:44.369429 5918 rebalancer_tool.cc:215] running whole-cluster rebalancing
W20250112 21:06:44.562063 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table dfd586c6ebfe48f39efc00f1bd759d08: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:44.562126 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table d647cf606b06442ebf35d3ebc20f7721: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:44.562139 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 7fdab271d7cd4decb290f197850245c2: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:44.562151 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5efd715f33104220b56c71d4cf8f47e6: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:44.562171 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5ecb633ba0ac40a5a150e8ed0dea146f: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:44.562182 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 3f42c0c2f1d146119b916da9e48b78d5: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:44.562199 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table fe7cb492a418403e8e71207bdfec4643: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:44.562217 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 7fdab271d7cd4decb290f197850245c2: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:44.562233 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5efd715f33104220b56c71d4cf8f47e6: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:44.562249 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table cef690fd88994bfb9c37221904a1eaef: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:44.562265 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table ef466625e9a14b348d82de0c47adfc6b: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:44.562279 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table f456baf32c6f4feba4f57d67d5f2dc09: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:44.562295 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table fe7cb492a418403e8e71207bdfec4643: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:44.562311 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table d647cf606b06442ebf35d3ebc20f7721: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:44.562326 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5ecb633ba0ac40a5a150e8ed0dea146f: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:44.562342 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table dfd586c6ebfe48f39efc00f1bd759d08: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 67a5a7038ced40ddb6f45fdbfd99825c
I20250112 21:06:44.595496 5918 rebalancer_tool.cc:1090] tablet 87835ddc19954987b0f17f6aa0370c10: 'fd8fa04957924e9f83afd61f9adfb455' -> '85a6e2a0c3b94dea8a094d396cfcac4b' move scheduled
I20250112 21:06:44.656749 5918 rebalancer_tool.cc:1090] tablet 979cf479af044430904e610adee50359: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '6f7f47d877a8407185013dba2959fddd' move scheduled
I20250112 21:06:44.702742 5918 rebalancer_tool.cc:1090] tablet 102b9dd051ff4a2a8da8ff4692e56256: '1d1923072e1c45778fa5afdcfda8144d' -> '38c8e8afd3f5411490f56ebf57aee109' move scheduled
I20250112 21:06:44.744822 5918 rebalancer_tool.cc:1090] tablet dd01159cf53642739a8a1dbcacbc933d: 'fd8fa04957924e9f83afd61f9adfb455' -> '67a5a7038ced40ddb6f45fdbfd99825c' move scheduled
I20250112 21:06:44.771404 5918 rebalancer_tool.cc:1090] tablet 74ff25b3286e45af8ddd38965062fca5: '2565f97e9cc54700ab2ce39aa7a546f8' -> '85a6e2a0c3b94dea8a094d396cfcac4b' move scheduled
I20250112 21:06:44.812943 5918 rebalancer_tool.cc:1090] tablet 7493a3e50b0e4fce955005b60a713348: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '67a5a7038ced40ddb6f45fdbfd99825c' move scheduled
I20250112 21:06:44.903790 5918 rebalancer_tool.cc:1090] tablet 90f8eabb6a0d4292953e3dec2efec97d: '1d1923072e1c45778fa5afdcfda8144d' -> '6f7f47d877a8407185013dba2959fddd' move scheduled
I20250112 21:06:44.927438 5918 rebalancer_tool.cc:1090] tablet 8880a7467c6a44da96f98c3cc368c9f0: '2565f97e9cc54700ab2ce39aa7a546f8' -> '38c8e8afd3f5411490f56ebf57aee109' move scheduled
I20250112 21:06:44.969307 5918 rebalancer_tool.cc:1090] tablet 1fb3deb52b8c4cceb1814858ea30e94f: 'fd8fa04957924e9f83afd61f9adfb455' -> '67a5a7038ced40ddb6f45fdbfd99825c' move scheduled
I20250112 21:06:44.999680 5918 rebalancer_tool.cc:1090] tablet c93763a09f7d4460b1874bf328fb06f5: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '85a6e2a0c3b94dea8a094d396cfcac4b' move scheduled
I20250112 21:06:45.014240 5918 rebalancer_tool.cc:1100] tablet 917b9e7ed0ed43efa0a60ff461e4f213: '1d1923072e1c45778fa5afdcfda8144d' -> '6f7f47d877a8407185013dba2959fddd' move ignored: Illegal state: tablet 917b9e7ed0ed43efa0a60ff461e4f213: replica 6f7f47d877a8407185013dba2959fddd is already present
I20250112 21:06:45.014307 5918 rebalancer_tool.cc:772] re-synchronizing cluster state
W20250112 21:06:45.125170 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table dfd586c6ebfe48f39efc00f1bd759d08: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.125221 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table d647cf606b06442ebf35d3ebc20f7721: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.125244 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5efd715f33104220b56c71d4cf8f47e6: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:45.125274 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table fe7cb492a418403e8e71207bdfec4643: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.125301 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 7fdab271d7cd4decb290f197850245c2: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.125329 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table f456baf32c6f4feba4f57d67d5f2dc09: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.125360 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5ecb633ba0ac40a5a150e8ed0dea146f: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.125386 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table cef690fd88994bfb9c37221904a1eaef: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.125411 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table ef466625e9a14b348d82de0c47adfc6b: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:45.125437 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table ef466625e9a14b348d82de0c47adfc6b: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.125465 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table dfd586c6ebfe48f39efc00f1bd759d08: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.125494 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table d647cf606b06442ebf35d3ebc20f7721: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:45.125523 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5efd715f33104220b56c71d4cf8f47e6: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.125551 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table fe7cb492a418403e8e71207bdfec4643: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.125581 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table f456baf32c6f4feba4f57d67d5f2dc09: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.125607 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 7fdab271d7cd4decb290f197850245c2: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.125634 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 3f42c0c2f1d146119b916da9e48b78d5: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.125660 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5ecb633ba0ac40a5a150e8ed0dea146f: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.125686 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table cef690fd88994bfb9c37221904a1eaef: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:45.158056 5918 rebalancer_tool.cc:1090] tablet 7fc2e530f44745948c4607c5f81b1d0e: '1d1923072e1c45778fa5afdcfda8144d' -> '38c8e8afd3f5411490f56ebf57aee109' move scheduled
I20250112 21:06:45.184084 5918 rebalancer_tool.cc:1090] tablet 0fce4710d0784d02991a6465522323d1: '1d1923072e1c45778fa5afdcfda8144d' -> '6f7f47d877a8407185013dba2959fddd' move scheduled
I20250112 21:06:45.282986 5918 rebalancer_tool.cc:1090] tablet c16b940a9e564da89ec84b46a6cf42d7: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '67a5a7038ced40ddb6f45fdbfd99825c' move scheduled
I20250112 21:06:45.293995 5918 rebalancer_tool.cc:1100] tablet 5da4fd48e1f24680babd4f1874b31b5f: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '38c8e8afd3f5411490f56ebf57aee109' move ignored: Illegal state: tablet 5da4fd48e1f24680babd4f1874b31b5f: replica 38c8e8afd3f5411490f56ebf57aee109 is already present
I20250112 21:06:45.294041 5918 rebalancer_tool.cc:772] re-synchronizing cluster state
W20250112 21:06:45.421947 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table dfd586c6ebfe48f39efc00f1bd759d08: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.422012 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5efd715f33104220b56c71d4cf8f47e6: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.422030 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 7fdab271d7cd4decb290f197850245c2: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.422053 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table d647cf606b06442ebf35d3ebc20f7721: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.422080 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table cef690fd88994bfb9c37221904a1eaef: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.422102 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5ecb633ba0ac40a5a150e8ed0dea146f: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 67a5a7038ced40ddb6f45fdbfd99825c
W20250112 21:06:45.422124 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table ef466625e9a14b348d82de0c47adfc6b: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:45.423430 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 3f42c0c2f1d146119b916da9e48b78d5: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.423482 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table f456baf32c6f4feba4f57d67d5f2dc09: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.423540 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table fe7cb492a418403e8e71207bdfec4643: could not find any suitable replica to move from server 1d1923072e1c45778fa5afdcfda8144d to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.423557 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table dfd586c6ebfe48f39efc00f1bd759d08: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 85a6e2a0c3b94dea8a094d396cfcac4b
W20250112 21:06:45.423573 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5efd715f33104220b56c71d4cf8f47e6: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.423589 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 7fdab271d7cd4decb290f197850245c2: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 38c8e8afd3f5411490f56ebf57aee109
W20250112 21:06:45.423616 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table d647cf606b06442ebf35d3ebc20f7721: could not find any suitable replica to move from server bd4e9b42a6de4ff7a2d954e2be23c606 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:45.423650 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table cef690fd88994bfb9c37221904a1eaef: could not find any suitable replica to move from server fd8fa04957924e9f83afd61f9adfb455 to server 85a6e2a0c3b94dea8a094d396cfcac4b
I20250112 21:06:47.027935 5918 rebalancer_tool.cc:1152] tablet 979cf479af044430904e610adee50359: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '6f7f47d877a8407185013dba2959fddd' move completed: OK
I20250112 21:06:47.219115 5918 rebalancer_tool.cc:1152] tablet 87835ddc19954987b0f17f6aa0370c10: 'fd8fa04957924e9f83afd61f9adfb455' -> '85a6e2a0c3b94dea8a094d396cfcac4b' move completed: OK
I20250112 21:06:48.032110 5918 rebalancer_tool.cc:1152] tablet dd01159cf53642739a8a1dbcacbc933d: 'fd8fa04957924e9f83afd61f9adfb455' -> '67a5a7038ced40ddb6f45fdbfd99825c' move completed: OK
I20250112 21:06:48.040144 5918 rebalancer_tool.cc:1152] tablet c16b940a9e564da89ec84b46a6cf42d7: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '67a5a7038ced40ddb6f45fdbfd99825c' move completed: OK
I20250112 21:06:48.393142 5918 rebalancer_tool.cc:1152] tablet c93763a09f7d4460b1874bf328fb06f5: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '85a6e2a0c3b94dea8a094d396cfcac4b' move completed: OK
I20250112 21:06:48.441320 5918 rebalancer_tool.cc:1152] tablet 1fb3deb52b8c4cceb1814858ea30e94f: 'fd8fa04957924e9f83afd61f9adfb455' -> '67a5a7038ced40ddb6f45fdbfd99825c' move completed: OK
I20250112 21:06:48.465615 5918 rebalancer_tool.cc:1152] tablet 90f8eabb6a0d4292953e3dec2efec97d: '1d1923072e1c45778fa5afdcfda8144d' -> '6f7f47d877a8407185013dba2959fddd' move completed: OK
I20250112 21:06:48.476329 5918 rebalancer_tool.cc:1152] tablet 8880a7467c6a44da96f98c3cc368c9f0: '2565f97e9cc54700ab2ce39aa7a546f8' -> '38c8e8afd3f5411490f56ebf57aee109' move completed: OK
I20250112 21:06:48.487691 5918 rebalancer_tool.cc:1152] tablet 7493a3e50b0e4fce955005b60a713348: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '67a5a7038ced40ddb6f45fdbfd99825c' move completed: OK
I20250112 21:06:48.506024 5918 rebalancer_tool.cc:1152] tablet 0fce4710d0784d02991a6465522323d1: '1d1923072e1c45778fa5afdcfda8144d' -> '6f7f47d877a8407185013dba2959fddd' move completed: OK
I20250112 21:06:48.517817 5918 rebalancer_tool.cc:1152] tablet 102b9dd051ff4a2a8da8ff4692e56256: '1d1923072e1c45778fa5afdcfda8144d' -> '38c8e8afd3f5411490f56ebf57aee109' move completed: OK
I20250112 21:06:48.533740 5918 rebalancer_tool.cc:1152] tablet 74ff25b3286e45af8ddd38965062fca5: '2565f97e9cc54700ab2ce39aa7a546f8' -> '85a6e2a0c3b94dea8a094d396cfcac4b' move completed: OK
I20250112 21:06:49.306156 5918 rebalancer_tool.cc:1152] tablet 7fc2e530f44745948c4607c5f81b1d0e: '1d1923072e1c45778fa5afdcfda8144d' -> '38c8e8afd3f5411490f56ebf57aee109' move completed: OK
W20250112 21:06:49.493726 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table cef690fd88994bfb9c37221904a1eaef: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server fd8fa04957924e9f83afd61f9adfb455
I20250112 21:06:49.541980 5918 rebalancer_tool.cc:1090] tablet 0d2ae40fed4b41218aa58c523d7d9785: '38c8e8afd3f5411490f56ebf57aee109' -> '2565f97e9cc54700ab2ce39aa7a546f8' move scheduled
I20250112 21:06:49.559511 5918 rebalancer_tool.cc:1090] tablet 87835ddc19954987b0f17f6aa0370c10: '67a5a7038ced40ddb6f45fdbfd99825c' -> '' move scheduled
I20250112 21:06:49.626511 5918 rebalancer_tool.cc:1100] tablet 102b9dd051ff4a2a8da8ff4692e56256: '2565f97e9cc54700ab2ce39aa7a546f8' -> '1d1923072e1c45778fa5afdcfda8144d' move ignored: Illegal state: tablet 102b9dd051ff4a2a8da8ff4692e56256: replica 1d1923072e1c45778fa5afdcfda8144d is already present
I20250112 21:06:49.626561 5918 rebalancer_tool.cc:772] re-synchronizing cluster state
W20250112 21:06:49.774853 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table 5ecb633ba0ac40a5a150e8ed0dea146f: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server 6f7f47d877a8407185013dba2959fddd
W20250112 21:06:49.774920 5918 rebalancer_tool.cc:1221] No replica could be moved this iteration: Not found: table fe7cb492a418403e8e71207bdfec4643: could not find any suitable replica to move from server 2565f97e9cc54700ab2ce39aa7a546f8 to server fd8fa04957924e9f83afd61f9adfb455
I20250112 21:06:49.783736 5918 rebalancer_tool.cc:1090] tablet b6ba48c8dc7e4ee485f85ee94aca256b: '2565f97e9cc54700ab2ce39aa7a546f8' -> '' move scheduled
I20250112 21:06:49.789621 5918 rebalancer_tool.cc:1090] tablet 917b9e7ed0ed43efa0a60ff461e4f213: '2565f97e9cc54700ab2ce39aa7a546f8' -> '' move scheduled
I20250112 21:06:50.297012 5918 rebalancer_tool.cc:1152] tablet 87835ddc19954987b0f17f6aa0370c10: '67a5a7038ced40ddb6f45fdbfd99825c' -> '' move completed: OK
I20250112 21:06:50.505632 5918 rebalancer_tool.cc:1152] tablet 917b9e7ed0ed43efa0a60ff461e4f213: '2565f97e9cc54700ab2ce39aa7a546f8' -> '' move completed: OK
I20250112 21:06:50.528411 5918 rebalancer_tool.cc:1152] tablet b6ba48c8dc7e4ee485f85ee94aca256b: '2565f97e9cc54700ab2ce39aa7a546f8' -> '' move completed: OK
I20250112 21:06:50.533797 5918 rebalancer_tool.cc:1152] tablet 0d2ae40fed4b41218aa58c523d7d9785: '38c8e8afd3f5411490f56ebf57aee109' -> '2565f97e9cc54700ab2ce39aa7a546f8' move completed: OK
I20250112 21:06:50.701990 5918 rebalancer_tool.cc:1090] tablet 7249078cad3e4bc3ab2b62d8887987df: '38c8e8afd3f5411490f56ebf57aee109' -> '1d1923072e1c45778fa5afdcfda8144d' move scheduled
I20250112 21:06:50.708832 5918 rebalancer_tool.cc:1090] tablet 979cf479af044430904e610adee50359: '2565f97e9cc54700ab2ce39aa7a546f8' -> '' move scheduled
I20250112 21:06:50.753175 5918 rebalancer_tool.cc:1090] tablet 345f4bf45767475ca481525f3d26a34a: '1d1923072e1c45778fa5afdcfda8144d' -> 'bd4e9b42a6de4ff7a2d954e2be23c606' move scheduled
I20250112 21:06:50.799278 5918 rebalancer_tool.cc:1090] tablet 0d2ae40fed4b41218aa58c523d7d9785: '85a6e2a0c3b94dea8a094d396cfcac4b' -> '6f7f47d877a8407185013dba2959fddd' move scheduled
I20250112 21:06:51.274135 5918 rebalancer_tool.cc:1139] tablet 979cf479af044430904e610adee50359: '2565f97e9cc54700ab2ce39aa7a546f8' -> '' move is abandoned: Not found: Server with UUID 2565f97e9cc54700ab2ce39aa7a546f8 not a member of the config. RaftConfig: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d1923072e1c45778fa5afdcfda8144d" member_type: VOTER last_known_addr { host: "127.26.82.68" port: 33569 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:51.391525 5918 rebalancer_tool.cc:772] re-synchronizing cluster state
I20250112 21:06:51.537381 5918 rebalancer_tool.cc:1090] tablet dd9176ff75364d81a5b3bdd0e7e10673: '67a5a7038ced40ddb6f45fdbfd99825c' -> '2565f97e9cc54700ab2ce39aa7a546f8' move scheduled
I20250112 21:06:51.547060 5918 rebalancer_tool.cc:1090] tablet 8880a7467c6a44da96f98c3cc368c9f0: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '' move scheduled
I20250112 21:06:51.587963 5918 rebalancer_tool.cc:1090] tablet de6b6ac6eeff431b9ac8cce47ea3a59e: '2565f97e9cc54700ab2ce39aa7a546f8' -> '38c8e8afd3f5411490f56ebf57aee109' move scheduled
I20250112 21:06:53.186769 5918 rebalancer_tool.cc:1152] tablet 0d2ae40fed4b41218aa58c523d7d9785: '85a6e2a0c3b94dea8a094d396cfcac4b' -> '6f7f47d877a8407185013dba2959fddd' move completed: OK
I20250112 21:06:53.190809 5918 rebalancer_tool.cc:1152] tablet 345f4bf45767475ca481525f3d26a34a: '1d1923072e1c45778fa5afdcfda8144d' -> 'bd4e9b42a6de4ff7a2d954e2be23c606' move completed: OK
I20250112 21:06:53.282967 5918 rebalancer_tool.cc:1152] tablet 8880a7467c6a44da96f98c3cc368c9f0: 'bd4e9b42a6de4ff7a2d954e2be23c606' -> '' move completed: OK
I20250112 21:06:53.286473 5918 rebalancer_tool.cc:1152] tablet dd9176ff75364d81a5b3bdd0e7e10673: '67a5a7038ced40ddb6f45fdbfd99825c' -> '2565f97e9cc54700ab2ce39aa7a546f8' move completed: OK
I20250112 21:06:53.290992 5918 rebalancer_tool.cc:1152] tablet de6b6ac6eeff431b9ac8cce47ea3a59e: '2565f97e9cc54700ab2ce39aa7a546f8' -> '38c8e8afd3f5411490f56ebf57aee109' move completed: OK
I20250112 21:06:53.907105 5918 rebalancer_tool.cc:1152] tablet 7249078cad3e4bc3ab2b62d8887987df: '38c8e8afd3f5411490f56ebf57aee109' -> '1d1923072e1c45778fa5afdcfda8144d' move completed: OK
I20250112 21:06:54.044982 5918 rebalancer_tool.cc:1090] tablet 8880a7467c6a44da96f98c3cc368c9f0: '85a6e2a0c3b94dea8a094d396cfcac4b' -> '1d1923072e1c45778fa5afdcfda8144d' move scheduled
I20250112 21:06:54.105551 5918 rebalancer_tool.cc:1090] tablet b6ba48c8dc7e4ee485f85ee94aca256b: '6f7f47d877a8407185013dba2959fddd' -> '2565f97e9cc54700ab2ce39aa7a546f8' move scheduled
I20250112 21:06:55.382716 5918 rebalancer_tool.cc:1152] tablet b6ba48c8dc7e4ee485f85ee94aca256b: '6f7f47d877a8407185013dba2959fddd' -> '2565f97e9cc54700ab2ce39aa7a546f8' move completed: OK
I20250112 21:06:55.483987 5918 rebalancer_tool.cc:1152] tablet 8880a7467c6a44da96f98c3cc368c9f0: '85a6e2a0c3b94dea8a094d396cfcac4b' -> '1d1923072e1c45778fa5afdcfda8144d' move completed: OK
*** Aborted at 1736716015 (unix time) try "date -d @1736716015" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGSEGV (@0x0) received by PID 5918 (TID 0x7f2c706c0480) from PID 0; stack trace: ***
@ 0x7f2c78f83980 (unknown) at ??:0
@ 0x7f2c786b38ab tcmalloc::ThreadCache::ReleaseToCentralCache() at ??:0
@ 0x7f2c786b3c75 tcmalloc::ThreadCache::ListTooLong() at ??:0
@ 0x560f038095ad std::unordered_map<>::~unordered_map() at ??:0
@ 0x7f2c76fca161 (unknown) at ??:0
@ 0x7f2c76fca25a exit at ??:0
@ 0x7f2c76fa8bfe __libc_start_main at ??:0
@ 0x560f0373bf5a _start at ??:0
F20250112 21:06:55.833845 5913 rebalancer_tool-test.cc:1274] Check failed: runner_func()
*** Check failure stack trace: ***
*** Aborted at 1736716015 (unix time) try "date -d @1736716015" if you are using GNU date ***
I20250112 21:06:55.838158 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.839264 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800006949) received by PID 26953 (TID 0x7ff556d03700) from PID 26953; stack trace: ***
@ 0x7ff568f4a980 (unknown) at ??:0
I20250112 21:06:55.843592 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.845821 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
@ 0x7ff5672f7fb7 gsignal at ??:0
@ 0x7ff5672f9921 abort at ??:0
I20250112 21:06:55.849431 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
@ 0x7ff568162dcd google::LogMessage::Fail() at ??:0
I20250112 21:06:55.852926 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
@ 0x7ff568166b93 google::LogMessage::SendToLog() at ??:0
@ 0x7ff5681627cc google::LogMessage::Flush() at ??:0
I20250112 21:06:55.856951 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
@ 0x7ff568163f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x55efa20cf065 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN4kudu5tools55ConcurrentRebalancersTest_TwoConcurrentRebalancers_Test8TestBodyEvEUlvE0_EEEEE6_M_runEv at ??:0
@ 0x7ff56797f6df (unknown) at ??:0
@ 0x7ff568f3f6db start_thread at ??:0
@ 0x7ff5673da71f clone at ??:0
I20250112 21:06:55.876377 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.882364 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.891057 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.892562 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.897190 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:55.900043 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:55.901607 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.905185 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.928881 5602 consensus_queue.cc:237] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.14, Last appended by leader: 13, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:55.929561 5864 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 3 index: 14. Preceding OpId from leader: term: 3 index: 15. (index mismatch)
I20250112 21:06:55.929811 20057 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20250112 21:06:55.930114 3807 raft_consensus.cc:1270] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 3 index: 14. Preceding OpId from leader: term: 3 index: 15. (index mismatch)
I20250112 21:06:55.930361 20057 consensus_queue.cc:1035] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20250112 21:06:55.930481 19747 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 LEADER]: Committing config change with OpId 3.15: config changed from index 13 to 15, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:55.932008 3665 catalog_manager.cc:5526] T de6b6ac6eeff431b9ac8cce47ea3a59e P 67a5a7038ced40ddb6f45fdbfd99825c reported cstate change: config changed from index 13 to 15, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New cstate: current_term: 3 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:55.932300 3807 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Committing config change with OpId 3.15: config changed from index 13 to 15, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:55.933444 5451 tablet_service.cc:1514] Processing DeleteTablet for tablet de6b6ac6eeff431b9ac8cce47ea3a59e with delete_type TABLET_DATA_TOMBSTONED (TS 6f7f47d877a8407185013dba2959fddd not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:51422
I20250112 21:06:55.933576 5864 raft_consensus.cc:2949] T de6b6ac6eeff431b9ac8cce47ea3a59e P 38c8e8afd3f5411490f56ebf57aee109 [term 3 FOLLOWER]: Committing config change with OpId 3.15: config changed from index 13 to 15, VOTER 6f7f47d877a8407185013dba2959fddd (127.26.82.69) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "38c8e8afd3f5411490f56ebf57aee109" member_type: VOTER last_known_addr { host: "127.26.82.72" port: 37429 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:55.933645 20234 tablet_replica.cc:331] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: stopping tablet replica
I20250112 21:06:55.933704 20234 raft_consensus.cc:2238] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:55.933739 20234 raft_consensus.cc:2267] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:55.934002 20234 ts_tablet_manager.cc:1905] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:55.935477 20234 ts_tablet_manager.cc:1918] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.14
I20250112 21:06:55.935529 20234 log.cc:1198] T de6b6ac6eeff431b9ac8cce47ea3a59e P 6f7f47d877a8407185013dba2959fddd: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-4/wal/wals/de6b6ac6eeff431b9ac8cce47ea3a59e
I20250112 21:06:55.936015 3651 catalog_manager.cc:4872] TS 6f7f47d877a8407185013dba2959fddd (127.26.82.69:40871): tablet de6b6ac6eeff431b9ac8cce47ea3a59e (table rebalance_test_table_3 [id=5ecb633ba0ac40a5a150e8ed0dea146f]) successfully deleted
I20250112 21:06:55.963364 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.965764 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.982993 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:55.988262 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:55.993722 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:56.000149 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:56.000589 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:56.009935 4178 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:56.010789 5844 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:56.013782 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:56.015281 5582 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20250112 21:06:56.018939 4049 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:56.030285 5713 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:56.031810 5451 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250112 21:06:56.040323 3917 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20250112 21:06:56.046015 3787 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20250112 21:06:56.066499 5602 consensus_queue.cc:237] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.14, Last appended by leader: 13, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } }
I20250112 21:06:56.066993 3807 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 3 index: 14. Preceding OpId from leader: term: 3 index: 15. (index mismatch)
I20250112 21:06:56.066991 5471 raft_consensus.cc:1270] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Refusing update from remote peer 67a5a7038ced40ddb6f45fdbfd99825c: Log matching property violated. Preceding OpId in replica: term: 3 index: 14. Preceding OpId from leader: term: 3 index: 15. (index mismatch)
I20250112 21:06:56.067230 19747 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20250112 21:06:56.067356 20057 consensus_queue.cc:1035] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20250112 21:06:56.068001 19088 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 67a5a7038ced40ddb6f45fdbfd99825c [term 3 LEADER]: Committing config change with OpId 3.15: config changed from index 13 to 15, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:56.068121 5471 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P 6f7f47d877a8407185013dba2959fddd [term 3 FOLLOWER]: Committing config change with OpId 3.15: config changed from index 13 to 15, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:56.068126 3807 raft_consensus.cc:2949] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 [term 3 FOLLOWER]: Committing config change with OpId 3.15: config changed from index 13 to 15, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:56.069275 3658 catalog_manager.cc:5526] T 87835ddc19954987b0f17f6aa0370c10 P bd4e9b42a6de4ff7a2d954e2be23c606 reported cstate change: config changed from index 13 to 15, VOTER 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71) evicted. New cstate: current_term: 3 leader_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bd4e9b42a6de4ff7a2d954e2be23c606" member_type: VOTER last_known_addr { host: "127.26.82.65" port: 42903 } } peers { permanent_uuid: "6f7f47d877a8407185013dba2959fddd" member_type: VOTER last_known_addr { host: "127.26.82.69" port: 40871 } } peers { permanent_uuid: "67a5a7038ced40ddb6f45fdbfd99825c" member_type: VOTER last_known_addr { host: "127.26.82.70" port: 43655 } } }
I20250112 21:06:56.070276 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet 87835ddc19954987b0f17f6aa0370c10 with delete_type TABLET_DATA_TOMBSTONED (TS 85a6e2a0c3b94dea8a094d396cfcac4b not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:56626
I20250112 21:06:56.070381 19919 tablet_replica.cc:331] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: stopping tablet replica
I20250112 21:06:56.070436 19919 raft_consensus.cc:2238] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 21:06:56.070468 19919 raft_consensus.cc:2267] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 21:06:56.070995 19919 ts_tablet_manager.cc:1905] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 21:06:56.072496 19919 ts_tablet_manager.cc:1918] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.14
I20250112 21:06:56.072546 19919 log.cc:1198] T 87835ddc19954987b0f17f6aa0370c10 P 85a6e2a0c3b94dea8a094d396cfcac4b: Deleting WAL directory at /tmp/dist-test-taskam1Qel/test-tmp/rebalancer_tool-test.3.ConcurrentRebalancersTest.TwoConcurrentRebalancers_0.1736715961716934-26953-0/raft_consensus-itest-cluster/ts-6/wal/wals/87835ddc19954987b0f17f6aa0370c10
I20250112 21:06:56.072965 3651 catalog_manager.cc:4872] TS 85a6e2a0c3b94dea8a094d396cfcac4b (127.26.82.71:37889): tablet 87835ddc19954987b0f17f6aa0370c10 (table rebalance_test_table_1 [id=fe7cb492a418403e8e71207bdfec4643]) successfully deleted